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FOREWORD 


The  Army’s  growing  reliance  on  digital  technologies  reinforces  and  extends  concerns  about 
skill  acquisition,  retention,  and  transfer.  Digital  information  technologies  affect  job  performance 
and  generate  new  training  issues  and  problems.  This  report  describes  how  digital  technologies  can 
help  solve  the  training  problems  they  create. 

This  report  recommends  an  integrated  set  of  methods  that  apply  digital  technologies,  such 
as  instrumented  command  and  control  systems  and  military  simulations,  to  train  the  skills  needed  to 
understand  and  maintain  a  pictorial  depiction  of  a  battlefield  situation  on  a  digital  display.  These 
method  recommendations  address  the  need  to  define,  communicate,  and  maintain  a  common 
picture  of  the  battlefield  situation  across  warfighters  and  supporters.  The  application  and 
integration  of  such  methods  should  help  the  Army  design  and  develop  digital  training  environments 
for  an  information  age  force. 

This  research  was  part  of  the  U.S.  Army  Research  Institute  (ARI)  for  the  Behavioral  and 
Social  Science's  program  to  train  the  force.  The  objective  of  ARI's  Future  Battlefield  Conditions 
(FBC)  team  at  Fort  Knox  is  to  enhance  soldier  preparedness  through  development  of  training  and 
evaluation  methods  to  meet  future  battlefield  conditions.  This  report  represents  efforts  for  Work 
Package  2228  Force  XXI  Training  Methods  and  Strategies  (FASTTRAIN).  Results  of  this  effort 
have  been  shared  with  other  ARI  units  engaged  in  similar  efforts  and  the  Fort  Knox  Test  and 
Experimentation  Coordination  Office  (TECO).  A  Memorandum  of  Agreement  (MOA)  with  the 
U.S.  Army  Armor  Center  (USAARMC)  supports  ARI's  research  on  training  requirements  and 
evaluation  methods.  This  MOA  is  titled  Manpower,  Personnel  and  Training  Research, 
Development,  Test,  and  Evaluation  for  the  Mounted  Forces,  16  October  1995. 

Z^AM.  SIMUTIS 
Technical  Director 
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APPLYING  DIGITAL  TECHNOLOGIES  TO  TRAINING:  A  FOCUS  ON  PICTORIAL 
COMMUNICATION 

EXECUTIVE  SUMMARY _ _ _ 


Research  Requirement: 

To  achieve  the  Army’s  information  age  objectives,  training  and  evaluation  methods  are 
required  to  improve  soldier-computer  performance.  Digital  technologies,  such  as  instrumented 
command  and  control  systems  and  military  simulations,  afford  unique  training  solutions.  The 
research  requirement  addressed  in  this  report  is  how  to  apply  digital  training  solutions  to  address 
training  and  evaluation  concerns. 

Procedure: 

This  report’s  approach  focused  on  the  Army  goal  to  apply  digital  technologies  to  provide 
a  common  picture  of  the  battlefield  to  warfighters  and  supporters,  and  improve  training.  To 
address  this  requirement,  Army  modernization  objectives,  efforts,  and  lessons  learned  were 
reviewed.  A  review  of  training  literature  identified  shortcomings  in  learning  theory  and  practice, 
and  barriers  that  restrict  the  application  of  technology  to  training.  This  review  stressed  the 
relatively  unique  ability  of  digital  technologies  to  pictorially  represent  battlefield  situations  and  the 
mental  models  of  trainees,  and  perceptually  link  these  representations  to  train  pictorial 
communication.  Digital  training  implications,  both  theoretical  and  practical,  were  examined  to 
balance  training  and  evaluation  efficiency  and  effectiveness. 

Findings: 

The  findings  are  a  recommended  set  of  training  and  evaluation  methods  that  apply  digital 
technologies  to  the  skills  needed  to  understand  and  maintain  a  pictorial  depiction  of  a  battlefield 
situation  on  a  digital  display.  These  method  recommendations  address  the  requirement  to  define, 
communicate,  and  maintain  a  common  picture  of  the  battlefield  shared  by  warfighters  and 
supporters.  For  each  of  these  requirements,  research  issues  are  raised  and  corresponding  training 
and  evaluation  methods  are  recommended  to  address  each  issue. 

The  method  recommendations  stress  an  empirical  approach  to  training  and  evaluation. 
Method  recommendations  repeatedly  examine  how  a  log  of  soldier-computer  interations  from 
instrumented  command  and  control  systems  can  automatically  provide  an  empirical  base  for 
assessing  performance  and  giving  feedback.  Conclusions  consider  how  integration  and 
implementation  of  the  training  and  research  methods  recommended  in  this  report,  in  concert  with 
digital  technologies,  might  promote  the  design  and  development  of  a  digital  training  environment. 
Key  training  and  evaluation  considerations  for  this  environment  focus  on  the  need  to  pictorially 
communicate  the  battlefield  situations  depicted  on  digital  displays. 


vii 


Utilization  of  Findings: 

The  methods  recommended  in  this  report  should  help  training  and  evaluation  researchers 
improve  soldier-computer  interaction  and  foster  the  skills  needed  to  understand  and  maintain  a 
pictorial  depiction  of  a  battlefield  situation  on  a  digital  display.  Method  implementation  will 
require  coordination  efforts  between  these  researchers  and  the  developers  of  training,  training 
simulation,  and  digital  systems  to  help  the  Army  design  and  develop  a  digital  training  environment 
that  provides  the  skills  required  for  an  information  age  force. 
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APPLYING  DIGITAL  TECHNOLOGIES  TO  TRAINING: 

A  FOCUS  ON  PICTORIAL  COMMUNICATION 

INTRODUCTION 

The  U.S.  Army’s  ongoing  modernization  plan  presumes  force  dominance  depends  on 
advanced  information  technology.  The  Army  envisions  a  digital  battlefield  on  which  all 
warfighters  and  supporters  are  networked  by  digital  information  systems,  referred  to  here  as 
Command,  Control,  Communication,  Computer  and  Intelligence  (C4I)  systems.  The  predicated 
military  capability  of  C4I  systems  is  epitomized  by  their  ability  to  portray  a  common  and  relevant 
picture  of  the  battlefield  situation.  To  achieve  the  Army's  information  age  objectives,  such  as  a 
common  picture  of  the  battlefield,  training  and  evaluation  methods  are  required  to  improve  soldier- 
computer  performance. 

To  address  this  requirement,  this  report  identifies  a  set  of  research  issues  and  provides 
corresponding  method  recommendations  to  help  realize  the  potential  of  C  I  systems.  These  issues 
and  methods  are  based  on  a  literature  review  summarized  in  the  background  and  method  sections. 
A  particular  focus,  perhaps  bias,  of  this  review  was  how  to  apply  digital  information  technology  to 
meet  C4I  training  and  evaluation  requirements.  In  other  words,  an  assumption  of  this  report's 
authors  and  much  of  the  literature  reviewed  is  that  computers  can  help  solve  many  of  the  training 
problems  they  create. 


This  report’s  approach  begins  by  considering  the  profound  impact  of  advanced  information 
systems,  particularly  digital  technologies,  on  job  performance  and  training  in  commercial  and 
military  organizations.  One  notable  impact  is  a  perceived  lag  in  the  application  of  advanced 
information  systems  to  training.  Another  impact  is  that  advanced  information  systems  designed  to 
aid  job  performance,  often  burden,  frustrate,  and  misinform  workers.  Reasons  for  such  negative 
impacts  are  reviewed  with  special  emphasis  on  reasons  related  to  inadequate  methods  for  training 
and  evaluating  computer-mediated  work. 

The  Army  requirement  for  training  and  evaluation  methods  directed  at  the  digital  battlefield 
of  the  future  is  also  reviewed.  The  Army's  expectations  for  a  digitally  equipped  force  and  lessons 
learned  bearing  on  these  expectations  from  related  Army  research  are  considered.  The  slant  of  this 
review  focuses  on  the  Army's  expectation  that  C4I  systems  should  provide  warfighters  and 
supporters  a  "common  picture"  of  the  battlefield.  This  common  picture  concept  is  discussed  as  a 
challenging  but  defining  objective  of  the  Army's  investment  in  advanced  information  systems.  To 
help  address  this  objective,  this  report  focuses  on  training  and  evaluation  methods  directed  at  the 
pictorial  communication  of  battlefield  situations. 

For  this  report,  pictorial  communication  means  that  a  C4I  display  should  convey  needed 
battlefield  information  in  picture  formats  the  user  can  satisfactorily  receive,  understand,  and  act  on. 
This  report's  pictorial  focus  stresses  the  Army's  expectation  that  C4I  systems  should  literally  depict 
a  picture  of  the  battlefield,  a  visual  image  or  illustration  of  the  battlefield  situation.  While  textual 
and  tabular  formats  may  be  needed  for  additional  detail,  a  pictorial  representation  of  the  battlefield 
situation  is  a  foremost  concern.  A  telling  expectation  is  that  a  C4I  display  will  convey  at  a  glance 
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the  user’s  battlefield  situation.  The  report's  focus  on  communication  stresses  that  when  the 
worker's  tool  is  an  information  technology,  communication  between  that  tool  and  the  worker  is 
critical  to  their  collaborative  performance,  such  as  creating  and  maintaining  a  picture  of  the 
battlefield  that  transfers  goal-relevant  information. 

The  intent  of  this  report,  therefore,  is  to  identify  and  capitalize  on  research  methods  that 
could  apply  advanced  information  systems  to  training  and  evaluation  efforts  directed  at  the  pictorial 
communication  of  battlefield  situations.  The  authors  readily  acknowledge  the  cited  work  of  others 
used  to  develop  the  methods  recommended.  The  contribution  of  this  report,  at  best,  is  to  document 
how  the  methods  reviewed  might  be  adapted  and  integrated  to  help  the  Army  meet  some  important 
C4I  training  and  evaluation  requirements. 

The  background  section  reviews  more  global  training  issues  and  methods.  In  the 
background,  for  example,  barriers  are  identified  that  deter  the  application  of  training  technology. 
These  barriers  include  a  project  versus  program  approach  to  training  development,  and  a  failure  to 
apply  more  unique  medium  capabilities.  Another  barrier  may  be  an  approach  to  training  that 
regards  training  content  as  an  external,  physical  reality  to  which  course  delivery  conditions  trainee 
performance.  In  contrast,  more  contemporary  training  approaches  are  considered  that  match 
training  content  to  what  trainees  already  know. 

The  background  section  examines  the  relatively  unique  ability  of  digital  technologies  to 
emulate  microworld  work  models,  such  as  a  battlefield  situation  depicted  on  a  C  I  display,  and  the 
mental  models  of  trainees  and  users.  Digital  training  methods  to  perceptually  link  these  models  are 
considered  for  training  pictorial  communication.  Finally,  some  practical  implications  of  digital 
training  are  examined,  to  balance  training  effectiveness  and  efficiency  concerns. 

The  method  section  reviews  and  recommends  training  and  evaluation  methods  for  three 
main  areas  of  research  directed  at  the  pictorial  communication  of  battlefield  situations  on  C4I 
displays.  These  research  areas  are  centered  on  common  picture  training  and  evaluation 
requirements:  define,  communicate,  and  maintain  a  common  picture  of  the  battlefield.  For  each  of 
these  areas,  a  set  of  research  issues  is  identified  and  corresponding  training  and  evaluation  methods 
are  recommended  to  help  address  the  issues  raised. 

The  methods  recommended  are  based  on  the  authors'  review  of  related  methods  and  the 
relatively  unique  capability  of  digital  information  systems,  such  as  instrumented  C4I  systems  and 
military  warfighting  simulations,  to  synthesize  training  and  evaluation.  The  method  discussion 
provides  a  more  detailed  description  of  the  methods  reviewed  or  directs  the  reader  to  descriptions 
available  in  the  literature.  The  method  recommendations  adapt  the  methods  reviewed  to  devise  a 
relatively  integrated  set  of  training  and  evaluation  methods  that  directly  address  the  research  issues 
under  each  of  three  research  areas  identified. 

Key  research  issues  to  define  the  common  picture  include  the  need  to:  develop  conceptual 
and  operational  definitions  of  the  common  picture  objective;  determine  user  information 
requirements;  assess  C4I  capabilities  against  user  information  requirements,  and,  assess 
performance  and  give  feedback.  Corresponding  method  recommendations  begin  with  a  working 
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definition  of  the  common  picture  based  on  relevant  aspects  of  Mission,  Enemy,  Terrain,  Troops, 
and  Time  (METT-T)  on  a  C4I  display.  Recommended  methods  for  developing  an  operational 
definition  stress  the  need  to  scope  the  problem.  Problem  bounding  methods  focus  on  information 
requirements  for  structured  battalion  and  brigade  exercises  at  duty  position  level  matched  against 
information  capabilities  for  a  battalion  and  brigade  level  C4I  system.  To  identify  user  information 
requirements,  method  recommendations  for  eliciting  user  and  expert  knowledge  stress  a  process 
trace  of  those  requirements  during  simulated  exercises.  Methods  for  assessing  C4I  capabilities 
against  user  information  requirements  that  result  in  C4I  display  codes  are  described.  Most 
importantly,  this  section  describes  how  instrumented  C4I  systems  can  automatically  analyze  the 
collaborative  process  required  to  develop  the  common  picture  depicted  on  users'  C4I  displays  at  any 
point  during  a  unit's  training  exercise. 

Key  research  issues  identified  under  communicate  the  common  picture  include  the  need  to: 
shape  C4I  representations  to  match  users’  mental  models  and,  subsequently,  shape  users'  mental 
models  to  match  C4I  representations.  Corresponding  method  recommendations  to  shape  C4I 
representations,  first  assess  how  well  C4I  systems  pictorially  communicate  meaningful  information 
to  the  user.  Second,  research  methods  are  recommended  to  assess  communication  tradeoffs 
between  the  Army's  analog  information  formats  versus  the  digital  formats  available  from  current 
and  future  C4I  systems.  Method  recommendations  to  shape  users'  mental  models  to  match  C4I 
representations  correspond  to  individual  training  methods  for  pattern  and  situation  recognition. 
These  methods  capitalize  on  the  ability  of  digital  technologies  to  provide  trainee  and  trainer  control 
over  transformations  in  the  tactical  patterns  and  battlefield  situations  depicted  on  C4I  displays.  A 
variety  of  methods  to  control  or  manipulate  training  presentation  formats  are  described  including:  a 
reduced  stimulus  environment,  animation,  perceptual  augmentation,  and  time  compression  and 
expansion.  Additionally,  the  recommended  methods  for  shaping  users'  mental  models  to  match  C4I 
representations  rely  on  documented  methods  for  developing  automatic  processing  capabilities  to 
recognize  tactical  patterns  and  situations. 

Key  research  issues  identified  under  maintain  the  common  picture  include  the  need  to: 
develop  a  model  of  information  exchanges  required  to  maintain  a  common  picture;  link  this 
communications  model  to  military  simulations  and  C4I  systems;  and,  apply  these  integrated  digital 
technologies  to  training  at  individual,  small  group  and  collective  levels.  Corresponding 
recommendations  first  describe  methods  for  developing  expert  and  machine  versions  of  a  common 
picture  communications  model.  Then,  training  implications  are  examined  based  on  linking  this 
communications  model  to  constructive,  virtual,  and  live  warfighting  simulations.  Finally,  training 
method  recommendations  for  individual,  small  group,  and  collective  levels  are  provided.  These 
recommendations  apply  these  integrated  digital  technologies  to  training  directed  at  managing 
information  to  maintain  a  common  picture  of  the  battlefield. 

The  conclusion  section  integrates  the  training  and  research  methods  recommended  in  this 
report,  in  concert  with  digital  technologies,  to  help  design  and  develop  a  digital  training 
environment  directed  at  pictorial  communication.  Conclusions  stress  that  method  integration  in 
virtual  simulation,  in  particular,  might  provide  an  environment  that  effectively  synthesizes  training 
and  evaluation.  Key  training  conclusions  consider  the  identification  of  training  requirements. 
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training  development  and  delivery,  and  training  analysis  and  feedback.  Key  evaluation  conclusions 
consider  machine,  soldier,  and  soldier-machine  performance. 

The  methods  recommended  in  this  report  should  help  training  and  evaluation  researchers 
improve  soldier-computer  interaction  to  pictorially  communicate  battlefield  situations.  Method 
implementation  will  require  coordinated  efforts  between  these  researchers  and  the  developers  of 
training,  training  simulation,  and  C4I  systems  to  achieve  the  Army's  information  age  objectives, 
such  as  a  common  picture  of  the  battlefield. 

BACKGROUND 

Advances  in  digital  information  technology  have  affected  job  performance  far  more  than 
job  training  (Army  Science  Board,  1995;  Baker  &  O’Neil,  1994).  Training  lags  are  a  paradox 
typical  of  technical  advances  intended  to  ease  our  life  and  work,  that  unintentionally  add 
complexity,  difficulty,  and  frustration  (Norman,  1988).  Progression  in  an  information  age  may 
require  applying  the  training  capability  of  information  technology  to  realize  advances  and 
overcome  unintended  consequences.  This  report’s  focus  on  how  to  apply  digital  training 
technologies  to  computer-mediated  work  assumes  that  many  of  the  required  answers  reside  in  the 
problem:  “Computers  not  only  create  training  problems,  but  can  be  used  to  solve  them”  (Patrick, 
1992,  p.  435). 

Computer-mediated  work  changes  the  underlying  skills,  abilities,  and  knowledge  required 
to  do  the  job  (Craiger,  1996).  While  many  industrial-age  jobs  required  repetitive  performance  of 
simpler  procedural  tasks  in  highly  structured  domains,  information-age  technologies  often  shift  job 
requirements  to  more  complex  and  uncertain  environments  that  require  information  processing  and 
decision-making  skills.  Particularly  in  their  early  stage,  information  technologies  may  increase 
requirements  on  a  worker’s  knowledge,  memory,  and  attention,  and  raise  overall  workload  (Cook 
&  Woods,  1996). 

Advances  in  computers  and  the  information  they  provide  may  also  redefine  a  worker’s  job 
domain.  On  the  one  hand,  by  automating  many  aspects  of  job  performance,  information 
technologies  may  consolidate  multiple  jobs  into  an  integrated  computer-based  workstation.  On  the 
other,  by  increasing  information  quantity  and  precision,  information  technologies  may  accentuate 
task  differentiation  and  fragment  a  job  domain  into  subspecialties  (Cook  &  Woods,  1996).  The 
ongoing  redefinition  of  work  from  task-based  products  to  team-based  processes  conflicts  with  the 
notion  of  “the  job”  as  a  fixed  bundle  of  tasks  (Cascio,  1995). 

In  the  military,  the  impact  of  information  technologies  on  job  performance  and  training  is  of 
particular  concern.  The  U.S.  Army  is  focused  on  information-age  warfare  and  the  exploitation  of 
information  technologies  to  maintain  a  dominant  force.  At  a  visionary  level,  this  is  a  force  of  cyber 
warriors  with  humans  and  computers  allied  as  a  joint  cognitive  system  (Cook  &  Woods,  1996). 
Wary  of  change  when  the  cost  of  errors  may  be  critical,  the  military  is  proactively  attempting  to 
identify  the  training  requirements  and  the  unintended  consequences  of  information  technologies 
(Alberts,  1996).  These  changes  not  only  affect  job  performance  and  training  at  individual  and 
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collective  levels,  but  the  relationships  among  organizations  and  workers,  such  as  commanders  and 
their  subordinates,  and  even  the  nature  of  military  operations. 

Investments  in  information  technology  underscore  the  need  to  leverage  computers  to 
improve  their  training  and  use.  Given  the  complexity  and  pace  of  contemporary  work,  advanced 
information  technologies  may  actually  be  essential  for  providing  training  and  support  technologies 
for  computer-based  work  (Fletcher,  1995).  For  some  time,  trainers  have  realized  that  computers 
afford  unique  solutions  to  many  important  training  problems  (Seltzer,  1971).  These  solutions 
include  providing  alternative  representations  of  information,  adaptive  control  of  training  delivery, 
and  instantaneous  feedback  to  the  responses  of  trainees.  In  particular,  “a  computer  is  virtually  the 
only  means  of  driving  a  complex  dynamic  simulation”  of  the  work  environment  to  provide  a 
meaningful  context  for  training  (Patrick,  1992,  p.  457). 

In  a  word,  our  progressive  alliance  with  and  reliance  on  digital  information  technologies 
may  be  irrevocable  (Negroponte,  1995).  In  response,  researchers  and  practitioners  are  redefining 
training  theory  and  design,  and  the  very  nature  of  the  training  environment.  Computer-mediated 
work  is  blurring  the  distinction  between  training  and  job  environments.  Electronic  Performance 
Support  Systems  (EPSS),  for  example,  provide  on-the-job  assistance  to  computer-based  workers 
(Cascio,  1995).  Embedded  training  systems  built  into  the  worker’s  tools  or  equipment,  such  as  a 
tank,  exemplify  how  training  is  becoming  part  of  the  real  world  (Morrison  &  Orlansky,  1997; 
Witmer,  1996).  Meanwhile,  the  ability  of  computers  to  simulate  a  worker’s  operational  setting  is 
bringing  the  real  world  to  the  training  environment.  The  progressive  fusion  of  the  job  and  training 
environment  induced  by  computer-mediated  work  poses  opportunities  and  challenges  in  computer- 
based  training. 

Training  is  key  to  achieving  the  potential  of  digital  technology  to  provide  a  meaningful 
portrayal  of  the  job  context,  such  as  a  battlefield  situation.  Training  is  also  key  to  avoiding 
technology  pitfalls.  For  example,  a  fundamental  training  issue  is  the  technical  paradox  that 
networked  computers  may  extend  a  worker’s  information  requirements  to  literally  global 
proportion  but  provide  only  a  window-size  interface  for  viewing  and  using  that  information. 

The  intuitive  appeal,  or  face  validity,  of  graphic  displays  represents  another  technology  pitfall:  the 
better  the  display,  the  greater  the  risk  that  it  will  lead  to  misperception  (Fadden,  Braune,  & 
Wiedemann,  1993). 

Cook  and  Woods  (1996)  provide  real-world  documentation  on  how  the  introduction  of 
integrated  information  systems  often  causes  serious  problems.  They  analyzed  expert  performance 
in  an  operating  room  after  it  was  equipped  with  an  integrated  display  that  was  designed  to  replace  a 
suite  of  stand-alone  patient  monitoring  systems.  Their  analysis  disclosed  numerous  limitations, 
particularly  during  high-tempo  work  periods.  They  concluded  that  displays  designed  to  provide 
more  precise  or  transformed  data  representations  often  mask  more  meaningful  data,  from  an  expert 
user’s  perspective.  They  cautioned  that  integrated  displays  designed  with  a  myriad  of 
reconfiguration  options  are  sometimes  “as  unwieldy  as  a  pocket-knife  with  50  tools”  (p.  612). 

This  report  draws  upon  available  literature  to  apply  digital  technologies  to  training  issues 
and  methods  for  the  Army’s  digital  tactical  displays.  The  intended  consequence  is  to  improve 
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training  directed  at  the  pictorial  communication  of  the  battlefield  situation  on  digital  tactical 
displays.  A  related  concern  is  to  avoid  unintended  consequences  that  might  imperil  soldiers  using 
tactical  displays  on  the  digital  battlefield. 

The  Digital  Battlefield 

The  U.S.  Army  is  embarked  on  a  force  modernization  plan  called  “Joint  Venture”  that 
presumes  future  force  dominance  depends  on  advanced  information  technologies  (Department  of 
the  Army  [DA],  1995).  Information  is  power,  and  a  precept  of  the  ongoing  revolution  in  land 
warfare  is  that  information  is  essential  to  combat  power  (Franks,  1994).  A  key  objective  directing 
this  modernization  effort  is  the  need  to  establish  deliberate  patterns  of  future  force  operations:  (a) 
project  the  force,  (b)  protect  the  force,  (c)  gain  information  dominance,  (d)  shape  the  battlespace, 

(e)  operate  decisively,  and  (f)  sustain  the  force  (DA,  1996a).  Advanced  information  technologies 
must  help  establish  these  patterns  on  the  digital  battlefield  of  the  future. 

The  concept  of  the  digital  battlefield  is  based  on  the  provision  of  networked  digital 
information  systems  to  all  soldiers  in  the  areas  of  combat,  combat  support,  and  combat  service 
support  (Decker,  1996).  These  digital  systems,  referred  to  in  this  report  as  Command,  Control, 
Communication,  Computer  and  Intelligence  (C4I)  Systems,  will  serve  as  the  tools  required  to 
perform  the  command  and  control  functions  directed  at  mission  accomplishment.  Command  and 
control  functions  and  tools,  whether  voice-based  or  digital  systems,  have  no  intrinsic  value  but  that 
which  is  derived  from  mission  success  (Alberts,  1996). 

In  sum,  digital  C4I  systems  are  expected  to  increase  the  quality  and  quantity  of  data 
available  and  assist  commanders  and  subordinates  by  processing  that  data  into  more  meaningful 
information.  Digitizing  the  battlefield  is  defined  as:  the  application  of  technology  to  acquire, 
exchange,  and  employ  timely  information  horizontally  and  vertically  integrated  to  create  a  common 
picture  of  the  battlefield  from  soldier  to  commander  (DA,  1998a,  p.  5). 

The  Common  Picture:  A  Focus  on  Pictorial  Communication 

The  predicated  military  capability  of  digital  information  systems  is  often  heuristically 
characterized  as  the  ability  to  portray  a  common  and  relevant  picture  of  the  battlefield  situation  to 
all  force  combatants  and  supporters  (DA,  1995).  This  “common  picture”  capability,  as  it  will  be 
referred  to  in  this  report,  stresses  the  Army’s  expectation  that  digital  systems  will  do  more  than 
provide  needed  information.  The  expectation  is  that  these  systems  will  transform  data  and 
information  to  pictorially  depict  meaningful  battlefield  situations  on  each  user’s  C4I  display.  A 
recurrent  and  telling  expectation  is  that  this  digital  system  interface  will  convey  the  user’s 
battlefield  situation  “at  a  glance”  (Bateman,  1997,  p.  49). 

As  a  concept,  the  common  picture  provides  a  meaningful  and  efficient  summation  of  a 
complex  objective,  a  communicative  power  akin  to  the  “same  sheet  of  music  expression.  To 
achieve  its  full  power,  however,  the  concept  must  become  a  reality.  At  present,  the  concept  of  the 
common  picture  is  not  fully  defined  in  an  operational  manner.  Conceptual  uncertainties  are  best 
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resolved  in  physical  manifestation,  such  as  notes  on  a  page  or  graphical  features  on  a  tactical 
display.  An  operational  definition  of  the  common  picture  should  specify  the  pictorial  elements 
required  on  a  C4I  display  to  depict  a  meaningful  battlefield  situation.  Notably,  the  elements 
depicted  must  be  relevant  to  the  user  of  that  display  or  the  user  s  duty  position,  and  the  picture  must 
reflect  meaningful  changes  on  a  dynamic  battlefield. 

As  a  construct,  the  common  picture  should  provide  an  empirically  verifiable  account  of  the 
pictorial  elements  required  for  this  picture  product.  Once  the  elements  are  defined,  the  construct 
should  define  an  empirically  verifiable  process  to  create  and  maintain  this  picture  product.  This 
process  would  include  both  individual  soldier  and  computer  inputs  to  the  product.  Moreover,  this 
process  would  include  collaborative  soldier  and  computer  inputs  across  all  the  combatant  and 
supporter  participants  in  the  battlefield  situation. 

Research  and  training  methods  are  needed,  however,  to  achieve  the  Army  s  common  picture 
objective.  Research  methods  should  provide  an  empirically  verifiable  account  of  the  pictorial 
elements  required  for  this  picture  product  and,  the  process  by  which  those  elements  are  created  and 
maintained.  Training  methods  should  then  address  the  identified  process  and  product  requirements, 
and  ensure  the  pictorial  elements  depicted  on  a  C4I  display  communicate  information  in  a  manner 
that  can  be  understood  and  acted  on. 

Digital  Technologies  and  Army  Training 


The  Army’s  Joint  Venture  effort  to  leverage  digital  technologies  includes  both  force  and 
training  development  programs  (DA,  1996a).  Force  development  efforts  for  digital  systems  are 
central  to  Joint  Venture.  The  impact  of  digital  technologies  on  Army  training  is  a  much  broader 
issue  that  impacts  the  basic  pillars  of  Army  training--the  institution,  the  unit  and  self-development. 
The  anticipated  capabilities  of  digital  technologies  to  deliver  computer-based  training  directly  to  the 
trainee  is  shifting  the  Army’s  training  strategy  from  balanced  pillars  to  an  emphasis  on  self¬ 
development  and  learner-controlled  training  (Hartzog  &  Canedy,  1997).  Digital  information 
networks  foster  this  shift:  “...the  net  will  move  more  of  the  focus  of  education  from  the  institution 
to  the  individual”  (Gates,  1996,  p.  217).  A  major  impetus  for  this  shift  is  the  anticipated  potential 
of  digital  technologies  to  provide  distance  learning  and  create  a  virtual  classroom  without  walls 
(Army  Science  Board,  1995).  There  is  a  growing  assumption  in  the  Army  that  digital  technologies 
can  deliver  training  to  any  individual  at  any  location. 

A  related  assumption  is  that  digital  technologies  can  situate  or  place  any  trainee  in  any 
simulated  work  context,  such  as  a  battlefield  situation.  Future  Army  training  will  be  based  on  an 
iterative  sequence  and  mix  of  warfighting  simulations— constructive,  virtual,  and  live— in  which 
soldiers  and  units  conduct  realistic  tactical  operations.  These  three  types  of  simulations  vary 
primarily  with  respect  to  their  realism.  Constructive  simulation  relies  on  simulated  operators, 
equipment,  and  situations;  virtual  simulation,  on  real  operators  in  simulated  equipment  and 
simulated  situations;  and  live  simulation,  on  real  operators  and  real  equipment  in  simulated 
situations  (Sikora  &  Coose,  1995).  Any  combination  of  these  three  levels  can  be  interactively 
linked  to  synthesize  a  training  environment  (Cosby,  1995).  This  report  stresses  the  use  of  virtual 
simulation,  a  relatively  powerful  digital  technology  for  Army  training  (Fletcher,  1994). 
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The  Army  faces  a  difficult  challenge  in  its  efforts  to  develop  a  digital  training  environment 
that  capitalizes  on  digital  technologies  such  as  distance  learning  and  networked  simulation.  Critics 
suggest  the  Army  has  not  fully  leveraged  these  capabilities  and  that  training  remains  costly  and 
simulations  remain  sub-optimal  (Army  Science  Board,  1 995).  Others  are  concerned  that  such 
technological  changes  will  make  current  training  support  systems  obsolete  and  result  in  significant 
training  deficiencies  (U.S.  Army  Training  Support  Center,  1996).  However,  the  Army  was  a 
forerunner  in  the  development  of  distance  learning  and  distributed  interactive  simulation 
technologies.  Despite  dwindling  resources  and  the  complexity  of  the  training  problem,  the  Army  is 
actively  committed  to  using  these  technologies  to  improve  training  (Hartzog  &  Canedy,  1996).  A 
notable  example  of  this  commitment  is  the  Advanced  Warfighting  Experiments  (AWE)  conducted 
by  the  Army  to  learn  about  the  digital  battlefield. 

The  Common  Picture:  Key  Lessons  Learned  from  the  AWEs 

The  Army’s  ongoing  Joint  Venture  effort  is  providing  valuable  lessons  and  identifying  key 
research  and  development  issues  directly  related  to  the  common  picture  objective.  The  AWE  called 
Focused  Dispatch  formally  codified  the  issue  “What  is  the  process  of  maintaining  the  common 
picture?”  at  the  battalion/task  force  level  (U.S.  Army  Armor  Center,  1996).  The  answer, 
unfortunately,  was  “swivel-chair  integration.”  Staff  personnel  manually  re-entered  digital  reports 
and  messages  between  noninteroperable  C4I  workstations.  With  no  tactical  display  to  integrate 
digital  communications,  tactical  operations  center  (TOC)  staff  members  had  to  glean  essential 
information  from  each  display  and  also  transfer  the  information  manually  to  a  common  situation 
display,  an  acetated  mapboard.  This  AWE’s  concern  with  the  common  picture  also  underscored 
the  need  for  a  systematic  analysis  of  the  informational  elements  required  to  depict  the  common 
picture  (Lickteig,  1996). 

For  a  subsequent  AWE,  Task  Force  (TF)  XXI,  TOC  personnel  and  individual  vehicle 
operators  were  provided  a  more  interoperable  suite  of  digital  C4I  systems.  In  addition,  some  of  the 
key  C4I  systems  provided  various  filters  that  allowed  the  users  to  customize  their  display.  Filter 
types  included  the  ability  to  select  a  desired  rate  for  updating  vehicle  locations,  and  “battlespace” 
filters  that  restricted  communications  received  to  a  user-specified  area  of  interest.  As  operators 
began  to  learn  how  to  use  these  filters,  the  issue  of  what  information  was  relevant  to  a  common 
picture  quickly  emerged  (R.  Munden,  personal  communication,  November  23, 1996).  For 
depictions  of  the  enemy  situation,  for  example,  company  commanders  and  their  subordinates 
required  information  at  the  individual  vehicle  level.  Higher  commanders,  such  as  the  brigade 
commander,  required  a  depiction  of  enemy  information  that  aggregated  individual  vehicles  into 
unit-size  icons.  A  key  research  question  raised  was  “how  to  provide  enemy  information  relevant  at 
each  operator  level  without  cluttering  the  tactical  display.” 

Many  of  the  lessons  learned  from  the  TF  XXI  AWE  concentrate  on  the  training  requirement 
to  help  warfighters  better  understand  the  battlefield  situations  depicted  on  their  tactical  displays. 

The  Commander  of  the  U.S.  Training  and  Doctrine  Command  (TRADOC)  concluded  this  AWE 
underscored  that  “Leadership  training  should  be  changed  to  help  commanders  adapt  to  and  trust  the 
rapid  computerized  representations  of  the  battlefield”  (Hartzog,  1997,  p.7).  In  support  of  that 
requirement,  for  example,  the  U.S.  Army  Intelligence  Center  proposed  a  Master  Analyst  Program 
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for  the  All-Source  Analysis  System  (Turner,  1997).  This  program  is  designed  to  provide 
commanders  specially  trained  intelligence  analysts  who  can  apply  and  integrate  digital  intelligence 
systems  and  products  into  the  mission. 

Such  AWE  lessons  learned  identify  some  of  the  key  training  research  issues  that  should  be 
addressed  for  the  Army’s  digital  tactical  displays  to  achieve  the  common  picture  objective,  the 
pictorial  communication  of  the  battlefield  situation. 

Learning  Theory  and  Instructional  Design 

If  digital  technologies  are  not  being  leveraged  for  training  and  instruction,  why  not?  What 
barriers  deter  the  development  and  application  of  training  technologies?  Some  identified  barriers  to 
technology  use  in  training  include;  the  entrenched  structure  of  training  and  educational  systems, 
the  short-sighted  concern  of  technology  developers;  and  a  project,  rather  than  program,  focus  in 
research  and  development  (Baker  &  O’Neil,  1994).  Other  barriers  identified  by  Patrick  (1992) 
point  more  directly  to  training  theory  and  design.  These  barriers  include  simplistic  assumptions 
about  knowledge,  and  an  emphasis  on  content  rather  than  presentation  formats  and  medium 
characteristics.  Patrick  also  emphasizes  the  lack  of  state-of-the-art  training  principles  and  poor 
dialogue  between  science  and  practice.  Perhaps  the  greatest  barriers  to  the  use  of  digital 
technologies  for  training  are  identified  in  the  following  statement:  . .  .the  bottleneck  remains  the 
same:  a  lack  of  expertise  in  how  to  create  good  training  programs,  coupled  with  the  fact  that  any 
training  development  is  still  a  labor-intensive  activity”  (Patrick,  1992,  p.  456). 

The  barriers  identified  to  the  effective  use  of  digital  technologies  in  training  are  substantial. 
Their  overall  magnitude  should  temper  expectations  for  immediate  and  immense  training 
improvements  from  technical  training  solutions.  However,  this  report  attempts  to  breach  two  of 
these  barriers,  in  particular.  First,  the  methods  recommended  represent  a  programmatic  approach  to 
the  application  of  digital  technologies  for  training.  Second,  these  methods  strive  to  improve 
training  format  by  exploiting  the  medium  characteristics  unique  to  digital  technologies  for  pictorial 
communication. 


Army  Training  Theory  and  Design 

Army  training  is  traditionally  based  on  Instructional  Systems  Development  (ISD) 
procedures  adapted  from  a  systems  approach  to  training  analysis,  design,  and  evaluation.  The  ISD 
model  guides  the  systematic  analysis  of  the  information  to  be  learned  and  the  tasks  to  be  performed 
and  imposes  an  external  structure  for  step-by-step  sequenced  delivery  (Konoske  &  Ellis,  1986). 
This  analysis  provides  a  coherent  framework  for  designing  training  content  based  on  prerequisite 
requirements.  Behavioral  learning  theory  complements  ISD  evaluation  by  decomposing 
complicated  tasks  into  a  series  of  measurable  behaviors  at  the  task  and  subtask  level  and 
consolidating  observable  “outputs”  into  training  outcomes. 

The  ISD  paradigm  for  training  reflects  its  industrial  age  origins  with  an  emphasis  on 
standardized,  mass  production  of  trainees.  The  standardized  procedures  of  ISD  are  geared  to  the 
simultaneous  training  of  multiple  trainees  on  the  same  training  content  in  the  same  amount  of  time. 
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Typically,  class  and  course  structures  define  the  traditional  educational  system  and  mimic  the 
physical  production  efficiencies  of  the  industrial  revolution.  Observable  performance  measures 
enable  normative  standards  for  assessing  training  proficiency  with  respect  to  institutional,  but  not 
necessarily  individual,  training  objectives.  Standardized  outcome  data  support  the  sorting,  versus 
learning,  of  trainees  for  subsequent  placement  or  fill  (Reigeluth,  1996). 

The  behavioral  theory  underlying  traditional  education  and  training  stresses  a  presentation 
and  response  paradigm  that  regards  the  trainee  as  a  passive  recipient  of  the  content  delivered. 
Content  is  viewed  as  an  external,  physical  reality  to  which  course  delivery  conditions  appropriate 
trainee  response.  Training  design  employs  rote  and  repetitive  learning  procedures  for  pairing  task 
conditions  with  task  performance.  “Classes”  of  trainees  are  viewed  as  equally  responsive  to  a 
course  treatment  that  results  in  equal  proficiency  (Osin  &  Lesgold,  1996). 

Contemporary  Learning  Theory  and  Design 

For  over  three  decades,  more  contemporary  learning  theory  and  design  approaches  have 
challenged  the  objectivist  foundation  of  behavioral  learning  theory  and  production-oriented  ISD 
principles  (Driskell,  Olsen,  Hays,  &  Mullen,  1995).  Three  of  the  primary  challengers  are  cognitive, 
constructive,  and  situated  learning  theorists.  ’While  these  theories  are  only  briefly  discussed  here, 
military  trainers  might  appreciate  the  learning  theory  analysis  of  traditional  and  contemporary  (but 
not  situated)  learning  theory  provided  by  Driskell  et  al.  (1995).  That  analysis  compares  learning 
theories  with  respect  to  how  learning  occurs,  factors  influencing  learning,  role  of  memory,  how 
transfer  occurs,  types  of  learning  supported,  and  the  structure  of  training.  For  a  more  detailed 
account  of  situated  learning  theory,  Druckman  and  Bjork’s  (1994)  review  of  training  transfer  is 
recommended. 

The  emergence  of  cognitive  psychology  contested  more  traditional  learning  theory  and 
design  by  focusing  on  mental  rather  than  physical  representations  of  systems  and  processes. 
Typically,  ISD  ignores  the  trainee’s  cognitive  framework,  including  prior  knowledge  and  mental 
representation  of  the  task  situation.  In  contrast,  cognitive  theory  addresses  how  the  mind  structures 
and  processes  information  acquisition,  storage,  integration,  and  retrieval  functions.  The  application 
of  cognitive  learning  theory  to  instructional  design  stresses  the  match  between  what  learners 
already  know  and  training  content  (Konoske  &  Ellis,  1986).  Cognitive  analyses  and  knowledge 
elicitation  methods  enable  the  development  of  mental  models  to  bridge  this  training  gap. 

Constructive  and  situated  learning  theories  assert  the  need  for  training  designs  that  make 
training  content  more  personally  meaningful  to  each  trainee.  Constructive  theorists  stress  that 
meaning  and  knowledge  exist  only  in  the  mind  and  must  be  actively  constructed  by  each  trainee 
(Jonassen,  1991).  Situated  learning  theorists  emphasize  that  critical  task  and  environmental 
features  and  cues  required  for  meaningful  training  are  only  available  in  the  actual  job  situation 
(Lave  &  Wenger,  1991).  The  two  theories  converge,  to  some  degree,  on  the  point  that  training 
should  occur  in  job  settings  that  represent  the  unique  and  dynamic  situations  that  typify  real  world 
job  situations  and  challenges. 
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Theoretical  Implications  for  Digital  Training 


To  derive  digital  training  implications  for  pictorial  communication,  some  key  theoretical 
aspects  of  contemporary  learning  theory  are  reviewed.  This  review  begins  by  considering  the 
emerging  role  of  the  worker’s  mental  model  and  its  external  complement,  the  microworld  model. 
Then,  the  role  that  perceptual  processes  play  in  linking  and  matching  these  models  is  considered. 

The  Worker’s  Mental  Model 


Contemporary  learning  theories  stress  that  the  cognitive  aspects  of  the  trainee,  such  as  prior 
knowledge  and  mental  representations,  should  be  matched  to  an  overarching  structure  of  the  task 
material  to  be  trained  (e.g.,  Driskell  et  al.,  1995).  A  cognitive  training  focus  goes  beyond  linear 
explanations  about  what  to  do,  by  also  addressing  the  how,  why,  when  and  where  explanations  for 
required  job  performance.  These  additional  explanations  provide  the  worker  a  basis  for  developing 
or  adapting  a  mental  model  of  the  external  job  environment.  Mental  models  are  essentially  internal 
representations  of  external  systems.  Mental  models  may  entail:  descriptions  of  system  purpose  and 
form;  explanations  of  system  functions  and  observed  system  states;  and  predictions  or  expectations 
of  future  system  states  (Fletcher,  1994). 

Cognitive  training  principles  stress  the  importance  of  mental  models  in  all  phases  of  skill 
acquisition— controlled,  associative,  and  automatic— and  for  both  part-  and  whole-task  training. 
During  the  initial  phases  of  training,  trainees  should  be  provided  a  simple  overview  of  the  system 
that  epitomizes,  versus  summarizes,  the  total  system  (Reigeluth,  Merrill,  Wilson,  &  Spiller,  1980). 
As  new  procedures  are  learned,  they  should  be  related  to  the  framework  of  that  mental  model.  An 
adequate  mental  model  is  expected  to  provide  a  coherent  reference  that  aids  the  recall  and 
application  of  task  information.  By  linking  trained  tasks  and  subtasks  to  a  model  of  the  job,  that 
knowledge  is  more  likely  to  be  usable  or  available  when  it  is  required  later,  on  the  job. 

The  collaborative  nature  of  teamwork  extends  training  concerns  to  the  issue  of  a  shared 
mental  model  among  workers.  Shared  mental  models  may  be  required  to  enhance  a  team’s 
coordination  of  job  requirements  and  improve  overall  team  performance.  For  general  training 
implications  related  to  shared  mental  models  see  Salas,  Prince,  Baker,  &  Shrestha  (1995). 

A  more  comprehensive  notion  of  a  mental  model  goes  beyond  system  states  or  situations  to 
include  the  dynamic  processes  underlying  changes  or  transitions  between  states  and  situations.  In 
this  sense,  a  mental  model  is  composed  of  a  connected  runnable  set  of  objects,  or  mental  entities 
(Williams,  Hollan,  &  Stevens,  1983).  A  runnable  mental  model,  for  example,  might  afford 
projections  of  a  future  battlefield  situation  based  on  the  current  situation.  Workers  may  use  their 
mental  model  to  identify  solutions  to  work  problems  by  changing  a  parameter  they  believe  defines 
a  state  in  order  to  achieve  a  different  state  representation.  Transitions  between  states  or  situations 
may  be  the  most  important  aspect  in  the  development  of  a  mental  model  (Kozma,  1991). 
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Microworld  Work  Models 


“Model”  is  an  imprecise  term  with  numerous  meanings,  as  discussed  by  Swezey  and 
Llaneras  (1997).  In  particular,  their  discussion  of  this  ambiguity  notes  that  simulation  and 
modeling  are  often  used  interchangeably  and  that  a  simulation  model  is  usually  a  computer-based 
representation  of  a  dynamic  process.  They  also  suggest  that  training-oriented  models  should  have 
several  unique  characteristics:  reliable  representation  of  a  real  situation,  control  over  represented 
characteristics,  and  deliberate  omission  of  certain  situational  characteristics. 

This  report's  integrated  concept  of  micro  world  models  and  the  worker’s  mental  model  is 
guided  particularly  by  the  writings  of  Kozma  (1991, 1994).  Kozma  asserts  that  learning  with  media 
is  a  complementary  process  in  which  representations  are  constructed  and  procedures  performed. 

His  work  suggests  that  digital  media  such  as  microworld  models  create  a  unique  training 
environment  that  supports  information  exchanges  and  interactions  between  coprocessors,  the 
trainee  and  the  computer.  The  links  between  these  joint  cognitive  systems  during  training  should 
mirror  their  links  on  the  job. 

The  advantages  and  varieties  of  training-oriented  models,  and  computer  models  in 
particular,  are  ably  documented  (Laughery  &  Corker,  1997;  Swezey  &  Llaneras,  1997).  This  work 
provides  numerous  examples  of  methods  for  applying  computer-based  models  to  a  wide  range  of 
training  domains.  This  discussion  is  restricted  to  two  microworld  models  directly  related  to  this 
report's  training  concerns. 

First,  a  model  for  a  man-machine  integrated  design  and  analysis  system  (MIDAS)  was 
developed  as  a  predictive  model  of  human  performance  to  aid  air  crew  system  designers  (Laughery 
&  Corker,  1997).  Although  MIDAS  was  not  designed  as  a  training  model,  it  was  intended  to  study 
the  differing  information  requirements  and  behaviors,  including  communications,  of  various  types 
of  workers  in  the  same  job  situation.  In  order  to  predict  the  collaborative  performance  of  flight 
crews  and  air  traffic  controllers,  MIDAS  includes  an  “updatable  world  representation”  and  a 
closed-loop  communications  model.  As  a  predictive  design  model,  method  applications  have 
focused  on  comparisons  between  the  model  and  actual  crew  and  controller  performance. 

Second,  a  team  model  training  (TMT)  methodology  by  Duncan  et  al.  (1996)  was  developed 
to  train  Information  Center  Anti-Air  Warfare  teams  on  Navy  ships.  The  model’s  design  for  team 
training  provides  several  methods  of  direct  relevance  to  training  individuals  and  small  groups 
engaged  in  collaborative  work.  In  particular,  TMT  includes  a  model  of  team  performance  and 
communications  designed  as  an  open-loop  model.  Applications  of  the  TMT  model,  therefore,  both 
passively  and  interactively  train  team  members.  Communications  modeled  by  TMT,  however,  are 
predominantly  verbal.  In  contrast,  a  model  based  on  digital  communications  between  C4I  systems 
and  their  displays  may  provide  a  training  medium  that  more  directly  reflects  the  job  setting.  With 
respect  to  training  analysis  and  feedback,  TMT  performance  assessments  were  predominantly 
based  on  experts’  direct  observation  and  post  hoc  review  of  videotapes. 
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Perceptual  Focus:  Matching  Internal  and  External  Models 


One  theme  re-emerging  from  more  contemporary  learning  theories  is  the  pivotal  role  of 
perception,  especially  the  visual  channel,  in  task  learning  and  performance.  Early  learning  theory 
grappled  with  the  issue  of  what  constitutes  or  represents  “identical  elements”  between  different  task 
situations  (Thorndike  &  Woodworth,  1901).  Nearly  a  century  later,  cognitive  theorists  contend  the 
identical  elements  theory  remains  viable  but  the  focus  has  shifted  from  external  observables  to 
internal  representations  or  mental  pictures  of  those  elements  (Singley  &  Anderson,  1989). 

This  renewed  perceptual  focus  in  training  and  performance  tracks  from  part-task  training  on 
feature  and  pattern  recognition  (Fischer  &  Geiwitz,  1996)  to  whole-task  training  such  as  situational 
recognition  in  complex,  decision-making  job  environments  (Klein,  1989).  The  goal  of  turning 
novices  into  experts  has  directed  the  attention  of  training  developers  to  a  cognitive  analysis  of  how 
experts  differentially  use  their  skills  to  solve  job-based  problems.  A  growing  consensus  about 
experts  is  that  their  solutions  are  based  on  salient  perceptual  cues  in  the  problem  situation,  and  that 
expertise  entails  automated  subskills  such  as  pattern  recognition  that  support  more  controlled  skill- 
based  solutions  and  decisions  (Fisk  &  Rogers,  1992). 

At  the  part-task  or  subtask  level,  Fischer  &  Geiwitz  (1996)  provide  a  cognitively-based 
approach  for  training  pattern  recognition  that  addresses  both  the  perceptual  aspects  and  meaning  of 
patterns.  While  their  training  approach  acknowledged  the  crucial  role  of  practice  and  feedback  for 
developing  pattern  recognition  skills,  it  was  paper-based.  Digital  training  technologies  provide  a 
unique  training  solution  for  the  drill  and  practice  required  to  develop  more  automated  processing 
skills,  such  as  pattern  recognition  (Salisbury,  1990).  More  automated  map  reading  skills  result  in 
faster  reaction  time,  less  variability,  and  less  susceptibility  to  information  overload  (Fisk  &  Eboch, 
1989). 


At  the  task  level,  a  unique  training  solution  that  digital  technologies  might  bring  to  pictorial 
communication,  centers  on  the  meaningful  representation  of  the  job  context.  Information  systems 
that  portray  a  rich  and  complex  context  can  be  an  effective  support  for  natural  decision-making 
(Klein,  1989).  A  tactical  display  can  provide  a  direct  perception  of  such  a  context,  the  battlefield 
situation,  if  its  representation  is  appropriate  both  for  acting  on  that  environment  and  for  thinking 
about  the  environment  at  the  level  the  user  chooses  (Rasmussen  &  Pejtersen,  1995). 

The  tactical  display  and  its  controls  should  perceptually  reflect  the  task  and  subtask 
structure  of  a  user’s  hierarchical  perception  of  task  requirements.  The  key  to  understanding 
mastery  of  a  complex  task,  such  as  chess  or  tactical  decision-making,  is  in  the  immediate  perceptual 
processes  where  the  task  is  structured  (Rasmussen  &  Pejtersen,  1995). 

Practical  Implications  for  Digital  Training 

Training  theory  directed  at  training  effectiveness  must  be  balanced  by  consideration  of 
training  efficiencies.  A  concern  common  to  practitioners  and  theorists  is  the  need  to  customize, 
versus  standardize,  approaches  to  training  (Reigeluth,  1996).  The  shift  from  behavioral  and  ISD- 
based  training  standardization  started  with  programmed  learning  in  the  40’s  (Patrick,  1992).  The 
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shift  was  energized  by  the  emphasis  of  cognitive  learning  theory  on  relating  internal  and  external 
representations.  Cognitive  analyses  of  how  the  mind  structures  and  processes  information  have 
ranged  from  categorical  differences  between  novice,  intermediate,  and  expert  performance  to 
individual  differences. 

Moreover,  many  contemporary  theorists  endorse  a  form  of  apprenticeship  or  mentoring 
which  tends  to  immerse  trainees  in  authentic  job  situations.  Real  or  highly  representative  job 
situations  provide  trainees  with  more  complete  and  natural  feedback  on  their  performance,  such  as 
resultant  changes,  and  corrective  feedback  to  eliminate  errors  and  misconceptions  (Montague  & 
Wulfeck,  1984).  While  the  emerging  requirement  to  customize  training  and  provide  realistic  job 
settings  may  increase  training  effectiveness,  it  will  increase  the  complexity  and  cost  of  training 
(Driskell  et  al.,  1995). 

The  effectiveness  of  customized,  or  individualized,  training  is  well  documented  (Regian  & 
Shute,  1992).  With  a  personal  instructor,  individuals  may  exceed  the  achievements  from  group- 
based  instruction  by  two  standard  deviations,  achieving  98th  percentile  rankings  (Bloom,  1984). 

The  cost  of  manually  customizing  training,  however,  is  increasingly  prohibitive,  particularly  for  the 
military  (Fletcher,  1995).  The  cumulative  effects  of  training  inefficiencies  are  sizable  when  you 
train  200,000  individuals  per  day  (Seidel  &  Perez,  1994),  and  that  estimate  includes  only  the 
institutional  (school  house)  pillar  of  military  training,  not  self-development  and  unit  training. 

Additional  costs  associated  with  the  assessment,  versus  the  delivery,  of  training  and 
performance  further  disparage  a  reliance  on  manual  methods.  Efforts  to  increase  the  reliability  of 
manual  methods  for  assessing  performance  may  cost  three  times  more  than  the  less  reliable 
methods  routinely  used  in  education  and  training  (Stecher  &  Klein,  1997).  Such  monetary 
estimates  do  not  include  the  learning  costs  incurred  with  manual  assessment  procedures  that  limit 
and  delay  training  feedback,  analysis,  and  evaluation.  Nor  do  they  include  the  operational  costs  for 
military  training  on  actual  weapon  systems,  1-3  orders  of  magnitude  more  than  simulation-based 
training  alternatives  (Fletcher,  1995). 

The  alleged  potential  of  digital  training  technologies,  however,  awaits  confirmation.  In  a 
critical  analysis  of  emerging  instructional  technologies,  Hooper  and  Hannafin  (1991)  state  that 
unique  causal  relationships  between  technology  and  learning  are  not  conclusively  established. 

They  stress  that  neither  instructional  design  nor  learning  may  differ  as  a  consequence  of 
technology,  in  a  fundamental  manner.  However,  their  analysis  identifies  how  advanced 
technologies  should  and  should  not  be  applied  to  five  cognitive  phases  fundamental  to  the  design  of 
instruction:  retrieving,  orienting,  presenting,  encoding,  and  sequencing.  Moreover,  they  conclude: 
“...emerging  instructional  technologies  undoubtedly  possess  the  potential  to  deliver  effective  and 
efficient  instruction  previously  impossible  or  at  least  impractical”  (p.  91). 

Contemporary  learning  theories  converge  in  endorsing  digital  training  technologies,  we 
conclude.  Although  they  diverge  in  many  key  aspects,  they  confront  a  common  dilemma:  the  goal 
of  training  customized  to  individual  and  job  differences  is  too  costly  with  manual  training 
technologies.  Advocates  of  digital  training  technologies  assert  that  computers  can  provide  the  most 
personalized  instruction  (Regian  &  Shute,  1 992).  They  propose  that  a  computer  s  almost  unlimited 
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capacity  for  information,  including  information  about  the  trainee,  may  make  it  more  aware  of  and 
responsive  to  the  individual  needs  of  each  trainee  than  humanly  possible.  Winn  (1989)  states  this 
position: 

...the  only  viable  way  to  make  decisions  about  instructional  strategies  that  meshes  with 
cognitive  theory  is  to  do  so  during  instruction  using  a  system  that  is  in  constant  dialogue 
with  the  student  and  is  capable  of  continuously  updating  information  about  student’s 
progress,  attitude,  expectations,  and  so  on.  (pp.  39-41) 

Expectations  of  the  computer  as  a  personalized  tutor,  master,  or  mentor  seem  premature 
(Driskell  et  ah,  1995).  A  more  somber  assessment  of  the  barriers  that  must  be  overcome  before  the 
computer  can  serve  as  an  effective  tutor  for  real-world,  and  often  inherently  complex,  job  domains 
was  provided  by  Patrick  (1992,  p.  472):  “the  enormity  of  this  task  is  beyond  our  current 
capabilities.” 


Background  Summary 

The  background  section  considered  the  profound  impact  of  advanced  information  systems 
on  job  performance  and  training  in  commercial  and  military  organizations.  Despite  the  negative 
impacts  such  systems  often  have  on  job  performance,  barriers  that  contribute  to  a  lag  in  training 
applications  persist.  Reasons  for  training  lags  and  negative  impacts  were  reviewed  with  special 
emphasis  on  reasons  related  to  inadequate  methods  for  training  and  evaluating  computer-mediated 
work. 


The  review  of  the  Army's  requirement  for  training  and  evaluation  methods  directed  at  the 
digital  battlefield  of  the  future  focused  on  the  Army's  expectation  that  C4I  systems  should  provide 
warfighters  and  supporters  a  common  picture  of  the  battlefield.  The  report's  emphasis  on  pictorial 
communication  stressed  the  Army's  expectation  that  C4I  systems  should  literally  depict  a  picture  of 
the  battlefield  that  communicates  needed  battlefield  information  in  picture  formats  the  user  can 
satisfactorily  receive,  understand,  and  act  on.  Research  methods  are  needed  to  define  the  common 
picture  and  training  methods  are  needed  to  achieve  it. 

The  remainder  of  this  background  section  attempted  to  identify  research  issues  and  methods 
pertinent  to  the  application  of  digital  technologies  to  train  pictorial  communication.  This  review 
stressed  the  ability  of  digital  information  systems  to  match  training  content  to  what  trainees  already 
know  by  emulating  microworld  work  models,  such  as  a  battlefield  situation  depicted  on  a  C4I 
display,  and  the  mental  models  of  trainees  and  users.  Digital  training  methods  to  perceptually  link 
these  models  were  considered  for  training  pictorial  communication.  Finally,  some  practical 
implications  of  digital  training  were  examined  in  order  to  balance  training  efficiency  and 
effectiveness. 

As  indicated,  the  intent  of  this  background  section  was  to  identify  documented  methods  that 
could  apply  advanced  information  systems  to  training  and  evaluation  efforts  directed  at  the  pictorial 
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communication  of  battlefield  situations.  The  subsequent  method  section  will  attempt  to  capitalize 
on  these  documented  methods.  The  method  section  provides  a  relatively  integrated  set  of  method 
recommendations  designed  to  help  the  Army  meet  some  important  C  I  training  and  evaluation 
requirements,  and  particularly  the  common  picture  objective. 

METHOD 

This  section  presents  an  integrated  set  of  research  and  training  method  recommendations 
directed  at  the  pictorial  communication  of  battlefield  situations  on  C  I  displays.  These  method 
recommendations  are  organized  into  three  main  areas  of  research  that  should  help  achieve  the 
common  picture  objective:  define,  communicate,  and  maintain  a  common  picture  of  the  battlefield. 
For  each  of  these  three  areas,  research  issues  are  identified  and  a  corresponding  set  of  method 
recommendations  is  provided  to  help  address  the  issues  raised.  Table  1  provides  an  overview  of  the 
research  areas,  issues,  and  method  recommendations  addressed  in  this  report. 

The  method  recommendations  are  based,  in  large  part,  on  methods  selected  from  the 
literature  reviewed.  Discussion,  therefore,  describes  the  recommended  methods  in  some  detail,  or 
directs  the  reader  to  others'  documented  descriptions.  The  Table  1  method  recommendations 
consolidate  and  adapt  a  relatively  integrated  set  of  training  and  evaluation  methods  to  more  directly 
address  the  research  issues  identified  in  this  report.  These  method  recommendations 
capitalize  on  the  relatively  unique  capability  of  digital  information  systems,  as  reviewed  in  the 
background  section,  to  synthesize  training  and  evaluation.  Overall,  method  recommendations  will 
stress  the  computer's  ability  to  create  pictorial  and  interactive  formats  for  training  and  feedback,  and 
to  empirically  assess  performance 


Define  the  Common  Picture 

This  section  raises  some  key  research  issues  that  should  be  addressed  to  define  the  common 
picture  requirement  and  translate  this  concept  into  an  empirical  and  useful  construct.  Initially, 
conceptual  and  operational  definitions  of  the  common  picture  must  be  developed  to  provide  a  basis 
for  realizing  this  Army  objective.  Corresponding  method  recommendations  begin  with  a  working 
definition  of  the  common  picture  based  on  relevant  aspects  of  METT-T.  A  rationale  for  this 
definition  is  provided  based  on  Army  training  and  doctrine,  and  its  concurrence  with  current  and 
future  system  C4I  requirements  is  addressed.  Recommended  methods  for  developing  an  operational 
definition  stress  the  need  to  first  scope  the  problem.  Problem  bounding  methods  focus  on  the 
information  requirements  for  structured  battalion  or  brigade  exercises  at  duty  position  level, 
matched  against  the  information  capabilities  for  a  battalion  or  brigade  level  C4I  system. 
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Table  1 


Overview  of  Research  Areas,  Issues,  and  Method  Recommendations 


Areas 

Issues 

Method  Recommendations 

Develop  a  Conceptual  Definition 

Relevant  METT-T  on  C4I  Display 

Develop  an  Operational  Definition 

Bound  the  Definitional  Problem 

Dpfinp 

Identify  Operational  Context 

FXXITP  Exercises 

the 

Identify  Operational  C4I  System 

FBCB2 

Common 

Identify  the  User  Level 

Duty  Position 

Picture 

Determine  Information  Requirements 

Knowledge  Elicitation 

Expert  Process  Tracing 

Assess  C4I  Information  Capabilities 

C4I  Display  Codes 

Assess  Performance,  Give  Feedback 

Instrument  C4I  Systems 

Shape  C4I  Representations 

C4I  Display  Decodes 

Alternative  Format  Comparisons 

Communicate 

Shape  Mental  Models 

Individual  Training 

the 

Pattern  Recognition 

Automatic  Processing 

Common 

Pattern  Transformation 

Picture 

Automatic  Processing 

Situation  Transformation 

Stimulus  Reduction 

Animation 

Perceptual  Augmentation 

Time  Compression  and  Expansion 

Situation  Recognition 

Develop  Communications  Model 

Common  Picture  Communications  Model 

Expert  and  Machine  Versions 

Integrate  Digital  Technologies 

Link  Simulations,  C4I  Systems  and  CPCM 

Maintain  the 

Constructive 

Common 

Virtual 

Picture 

Live 

Develop  Multi-Level  Training 

Individual  Training 

Observer  and  Performer  Mode 

Small  Group  Training 

Collective  Training 

Pre  and  Post  Exercise 

Note.  METT-T  =  Mission,  Enemy,  Terrain,  Troops,  and  Time;  FXXTP  =  Force  XXI  Training  Program; 
C4I  =  Command,  Control,  Communications,  Computers  and  Intelligence;  CPCM  =  Common  Picture 
Communications  Model;  and,FBCB2  =  Force  XXI  Battle  Command — Brigade  and  Below. 
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Once  the  definitional  problem  is  scoped  to  a  more  workable  size,  the  issue  of  determining 
user  information  requirements  is  addressed.  To  identify  user  information  requirements,  methods 
for  eliciting  user  and  expert  knowledge  in  commercial  and  military  settings  are  briefly  reviewed. 
This  review  provides  a  basis  for  selecting  and  adapting  the  methods  recommended  for  further 
defining  the  common  picture  requirement.  Alternative  methods  are  recommended  to  address  more 
pragmatic  and  thorough  knowledge  elicitation  requirements.  This  report  s  primary 
recommendation,  however,  is  a  form  of  process  tracing  that  elicits  experts  knowledge  during  actual 
exercises.  Once  information  requirements  are  determined,  the  issue  of  assessing  C  I  capabilities 
against  those  requirements  is  raised.  An  evaluation  method  is  recommended  to  address  this  issue 
that  focuses  directly  on  the  pictorial  information  provided  by  a  C4I  display. 

Finally,  this  section  raises  the  need  for  instrumented  C4I  systems  that  provide  a  log  of 
human-computer  interaction.  Discussion  of  this  issue  underscores  the  need  for  automated  analysis 
and  feedback  on  the  information  tasks  that  are  a  basis  of  command  and  control  performance. 
Corresponding  method  recommendations  describe  how  instrumented  C4I  systems  can  automatically 
analyze  the  collaborative  process  required  to  develop  the  common  picture  across  a  unit  and,  the 
common  picture  product  depicted  on  users'  C4I  displays  at  any  point  during  a  unit  s  training 
exercise.  A  working  example  of  such  methods  is  described  called  automated  pictorial  comparisons, 
based  on  normative  or  criterion  standards  of  a  common  picture. 

Develop  a  Conceptual  Definition:  A  METT-T  Approach 

The  C4I  display  and  interface  is  a  primary  point,  or  means,  of  interaction  between  the 
soldier  and  the  battlefield  situation.  The  display’s  representations  should  enable  actions  and 
thought  on  the  depicted  environment,  at  the  level  the  user  chooses  (Rasmussen  &  Pejtersen,  1995). 
Methods  for  defining  the  common  picture,  therefore,  should  originate  with  the  user’s  mental  picture 
of  a  battlefield  situation. 

Key  factors  comprising  a  battlefield  situation  are  traditionally  and  heuristically  identified  as 
METT-T.  For  many  years,  METT-T  has  provided  a  coherent  training  structure  for  essential  types 
of  battlefield  information:  who,  what,  where,  how,  when,  and  why.  This  METT-T  organization  of 
battlefield  information  remains  central  to  current  doctrinal  concepts  such  as  Battlefield 
Visualization  and  Battle  Command.  For  example,  guidelines  for  staff  training  state  “the  major 
components  of  commander’s  visualization  are  based  on  METT-T  factors  (DA,  1997b,  p.  H). 

More  specific  subsets  of  information  requirements,  such  as  commander’s  critical 
information  requirements  (CCIR),  reflect  a  similar  organization.  “The  commander  must  review  his 
critical  information  requirements  for  each  operation  based  on  METT-T  factors”  (DA,  1997b,  p.  I- 
2).  Notably,  the  three  types  of  CCIR  are  also  METT-T  based:  priority  information  requirements 
(PIR)  relate  to  enemy  and  terrain;  friendly  forces  information  requirements  (FFIR)  relate  to  the 
combat  capabilities  of  own  and  adjacent  units  or  troops;  and,  essential  elements  of  friendly 
information  (EEFI)  relate  to  protecting  troops  from  the  enemy’s  information  gathering  systems 
(DA,  1996c).  Across  all  three  types,  critical  information  is  that  information  that  directly  affects  the 
successful  execution  of  operations. 
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The  METT-T  organization  of  battlefield  information  also  concurs  with  the  Army’s  overall 
approach  to  C4I  system  design  and  assessment.  “The  Army  Battle  Command  System  (ABCS)  must 
provide  the  flexibility  to  tailor  and  access  the  ABCS  common  database  to  meet  a  specific  set  of 
critical  information  needs  as  defined  by  METT-TC”1  (DA,  1998b,  p.  15).  The  ABCS  requirement 
for  a  METT-T  based  approach  is  clear  and  compelling:  this  capstone  system  defines  the 
requirements  for  all  current  and future  Army  C4I  systems. 

Defining  the  domain  and  content  of  user  information  requirements  is  an  essential  issue  to 
improving  training  and  evaluation  methods  for  C4I-based  operations.  Admittedly,  other 
informational  structures  or  frameworks  than  METT-T  might  be  used  to  define  the  infonnation 
requirements  for  C4I  displays,  to  include  tasks,  functions,  or  battlefield  operating  systems. 

Strategies  to  restrict  the  domain  of  requirements  might  also  prove  useful.  Such  strategies  might 
target  the  information  required  for  selected  processes,  products,  or  outcomes  related  to  unit 
performance;  or  address  a  subset  of  information  requirements,  such  as  CCIR.  While  future  research 
efforts  might  carefully  consider  the  tradeoffs  associated  with  alternative  informational  domain 
structures,  this  report’s  methods  will  use  the  METT-T  approach  for  defining  and  assessing  user 
information  requirements  for  battlefield  depictions  on  C4I  displays. 

A  Working  Conceptual  Definition  of  the  Common  Picture 

For  this  report,  a  common  picture  of  the  battlefield  is  defined  as  the  relevant  METT-T 
information  pictorially  communicated  by  a  C4I  display.  More  detailed  discussion  of  this  definition 
is  provided  in  the  following  section  that  addresses  developing  an  operational  definition  of  the 
common  picture.  Three  aspects  of  this  definition  are  noted.  First  the  specification  of  METT-T 
battlefield  factors  reflects  the  Army’s  doctrine  and  training  literature,  and  capstone  requirement  for 
C4I  systems,  as  previously  discussed.  Second,  this  definition’s  focus  on  pictorial  formats 
underscores  the  assumption  that  a  C4I  display  should  literally  portray  a  visual  image  of  the 
situational  elements  that  comprise  a  battlefield  situation.  Third,  the  emphasis  on  communication 
stresses  the  role  of  a  C4I  display  in  promoting  an  understanding  of  the  battlefield  situation,  a  role 
reinforced  through  METT-T  structure  and  pictorial  formats. 

Develop  an  Operational  Definition:  Bound  the  Problem 

To  provide  an  empirical  basis  for  training  and  assessing  performance,  the  common  picture 
concept  must  be  operationally  defined.  An  operational  definition  of  the  common  picture  should 
specify  what  pictorial  elements  are  required  on  a  C4I  display  to  depict  a  meaningful  battlefield 
situation,  and  the  process  by  which  those  elements  are  generated.  At  a  conceptual  level  this  may 
seem  fairly  easy,  as  indicated  by  recurrent  assertions  that  a  soldier  needs  to  know  the  location  of 
enemy  and  friendly  units  and  self  to  have  situational  awareness.  The  working  definition  proposed 
herein  for  the  common  picture,  however,  entails  a  more  comprehensive  and  challenging 


1  METT-TC  is  a  recent  extension  of  METT-T  that  adds  a  civilian  (C)  factor.  Civilian  considerations,  however,  are  not 
yet  reflected  in  most  Army  training  and  doctrine  publications  and  they  will  require  additional  specification.  This 
civilian  factor  is,  therefore,  not  addressed  in  this  report. 
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requirement.  Clearly,  this  definition  extends  the  requirement  beyond  Enemy  and  Friendly 
information,  to  include  Mission,  Terrain  and  Time  factors. 

Additionally,  more  detailed  specification  of  the  information  related  to  each  METT-T  factor 
is  also  required.  For  example,  soldiers  generally  need  to  know  more  about  the  enemy  than  its 
location.  Elements  of  required  information  in  an  enemy  Spot  report  format,  for  example,  include: 
size,  activity,  location,  unit  type,  time,  and  equipment  (SALUTE).  Additionally,  soldiers  may  need 
to  know  what  is  the  intent  or  disposition  of  an  enemy  unit  and  what  are  the  enemy's  capabilities. 
Table  2  provides  some  notional  informational  elements  that  soldiers  might  require  for  each  METT- 
T  factor.  This  list  is  provided  as  a  working  example  of  the  informational  detail  that  may  be 
required  to  provide  a  meaningful  picture  of  the  battlefield  situation. 

Moreover,  this  report's  conceptual  definition  stresses  that  required  METT-T  information 
should  be  pictorially  communicated  by  a  C4I  display.  The  emphasis  on  pictorial  formats  raises  the 
need  to  literally  depict  a  picture  of  the  battlefield  situation  on  a  C4I  display.  While  textual  and 
tabular  formats  may  be  needed  for  additional  detail,  a  pictorial  representation  of  the  physical 
situation  is  the  operational  standard  by  which  the  common  picture  requirement  should  be  assessed. 
The  emphasis  on  a  picture  that  communicates  stresses  that  a  Cl  display  should  convey  needed 
battlefield  information  in  picture  formats  the  user  can  satisfactorily  receive,  understand,  and  act  on. 

Research  and  training  methods  to  establish  an  operational  definition  of  the  common  picture 
are  recommended  and  discussed  throughout  this  section.  The  more  immediate  concern,  however,  is 
to  scope  or  bound  the  definitional  problem  to  a  manageable  level.  Three  key  bounding  issues  are 
identified,  and  method  recommendations  for  addressing  these  issues  are  provided.  These 
recommendations  bound  the  definitional  problem  by  identifying  an  operational  context,  an 
operational  Cl  system,  and  an  operator  level,  namely  duty  position.  Alternative  recommendations 
may  certainly  be  warranted,  such  as  a  different  operational  context  or  a  mix  of  different  Cl 
systems.  The  current  intent  is  merely  to  illustrate  how  the  definitional  problem  might  be  bounded, 
and  establish  boundary  examples  that  will  be  used  in  this  report. 

Identify  an  Operational  Context 

The  operational  context  for  the  U.  S.  Army  is  changing  and  expanding  as  it  becomes  an 
information  age  force  (DA,  1996a).  Efforts  to  identify  and  define  future  operational  contexts  are  an 
ongoing  effort.  The  immediate  concern,  however,  is  to  select  an  operational  context  that  is  both 
representative  and  relatively  well  defined  to  bound  the  definitional  problem.  At  best,  the 
operational  definition  of  a  common  picture  that  emerges  from  the  context  identified  here  might 
provide  a  solid  foundation  for  future  training  and  evaluation  efforts. 

A  key  part  of  the  Army’s  ongoing  effort  to  improve  training  is  the  Force  XXI  Training 
Program  (FXXITP).  A  basic  goal  of  this  program  is  to  provide  structured,  simulation-based 
training  exercises  to  improve  force  readiness.  A  forerunner  of  the  FXXITP  was  the  Virtual 
Training  Program  (VTP)  developed  by  the  Army  Research  Institute  at  Fort  Knox  using  virtual  and 
constructive  simulations  at  Fort  Knox  (Campbell,  Campbell,  Sanders,  Flynn,  &  Myers,  1995).  The 
VTP  methodology  is  rooted  in  a  set  of  structured  and  tactically  realistic  scenarios. 
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Table  2 


Notional  METT-T  Information  Requirements 


Mission  Enemy 


Terrain  Troops 


Time 


WARNO 
OPORD 
OPS  Overlay 
CDR’s  Intent 
COA 
FRAGO 


Location 
Composition 
Disposition 
Ability  to  See 
Ability  to  Move 
Ability  to  Shoot 
Ability  to  Communicate 
Ability  to  Sustain 


Observation 

Cover 

Concealment 
Obstacles 
Key  Terrain 
Approach  Avenues 
Weather 


Location 
Organization 
Ability  to  See 
Ability  to  Move 
Ability  to  Shoot 


Plan 

War  Game 
Prepare 
Rehearse 
Execute 


Ability  to  Communicate  Sync  Execute 
Ability  to  Sustain 


Note:  WARNO  =  Warning  Order;  OPORD  =  Operations  Order;  OPS  =  Operations;  CDR’s  = 
Commander’s;  COA  =  Course  of  Action;  FRAGO  =  Fragmentary  Order;  Sync  =  Synchronization 

Central  to  the  VTP’s  structure  is  the  detailed  specification  of  scenario  factors  such  as  the  location, 
organization,  status  and  disposition  of  friendly  and  enemy  forces,  and  the  delineation  of  scripted 
events  during  the  exercise.  The  VTP  scenarios  are  task-based  and  primarily  address  the  execution 
phase  of  a  mission.  They  also  include  planning  phase  stimulus  materials,  however,  such  as 
operations  orders  and  overlays. 

The  VTP  methodology  was  adopted,  in  large  part,  by  the  FXXITP.  The  original  platoon  to 
battalion  scenarios  were  based  on  the  National  Training  Center  (NTC)  and  addressed  three  basic 
Armor  and  Mechanized  Infantry  missions:  Defense  in  Sector,  Movement  to  Contact,  and 
Deliberate  Attack.  This  operational  context  has  been  routinely  tested  and  upgraded  during 
expansion  to  brigade-level  exercises  (Campbell,  Graves,  Deter,  &  Quinkert,  1998).  This  expansion 
from  battalion  to  brigade-level  training  exercises  also  recognized  the  need  to  address  the  planning, 
preparation,  and  execution  phases  of  a  mission.  Additionally,  the  brigade  training  exercises  include 
detailed  vignettes  tailored  to  the  unique  informational  requirements  of  a  wide  range  of  combat  and 
support  duty  positions.  By  design,  the  FXXITP  exercises  are  tailored  to  the  constraints  and 
capabilities  of  virtual  and  constructive  warfighting  simulations. 

The  recommended  operational  context  for  defining  the  common  picture  on  a  C4I  display, 
therefore,  is  the  training  exercises  developed  by  the  FXXITP.  More  specifically,  this  context 
includes  battalion  and  brigade  level  training  exercises  that  address  three  representative  and 
relatively  well  defined  missions:  Area  Defense,  Movement  to  Contact,  and  Deliberate  Attack.  As 
noted,  alternative  context  recommendations  for  different  missions,  locations,  and  organizations  may 
be  warranted  to  further  identify  information  requirements  for  a  common  picture. 
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Identify  an  Operational  C4I  System 


The  current  operational  C4I  systems  of  the  U.  S.  Army  represent  an  uneven  mix  of 
capability  and  compatibility,  as  noted  in  the  background  discussion  of  AWE  lessons  learned.  Such 
lessons  have  resulted  in  a  capstone  requirement  for  a  C4I  system  of  systems  called  the  Army  Battle 
Command  System,  or  ABCS.  More  specifically,  the  ABCS  document  encompasses  seven  major 
C4I  systems  and  an  unspecified  number  of  legacy  systems.  The  purpose  of  the  ABCS  requirement 
is  to  ensure  common  C4I  system  capabilities,  including  compatibility,  that  result  in  "a  user  defined 
relevant  common  operational  picture  of  the  battlefield  (DA,  1 998b,  p.  4).  Realistically,  this 
capstone  requirement  document  describes  itself  as  a  living  requirements  document. 

Prior  identification  of  a  battalion  and  brigade  operational  context  directs  identification  of  an 
appropriate  operational  C4I  system.  A  C4I  system  called  Force  XXI  Battle  Command— Brigade  and 
Below  (FBCB2)  is  one  of  the  seven  components  of  ABCS.  The  FBCB2  system  is  the  primary  C4I 
system  for  digitally  linking  brigade  and  battalion  echelons  to  soldier  and  platform  level  (U.S.  Army 
Armor  Center,  1997).  The  method  recommendation,  therefore,  identifies  FBCB2  as  the  operational 
C4I  system  for  determining  the  pictorial  informational  elements  required  for  a  common  picture  at 
brigade  and  battalion  levels.  The  FBCB2  system  will  serve  as  a  working  example  to  apply  the 
methods  recommended  in  this  report. 

Identify  the  Operator  or  User  Level 

Information  relevance  is  an  issue  critical  to  the  common  picture  concept  and  construct. 
However,  the  relevance  of  information  is  user  dependent,  particularly  from  a  contemporary  learning 
perspective.  In  the  military,  the  relevance  of  information  is  directly  linked  to  the  user’s  duty 
position,  such  as  the  CCIR  that  focus  on  the  commander's  duty  position.  Every  warfighter  and 
supporter  duty  position  has  unique  informational  requirements. 

Since  different  duty  positions  result  in  different  information  requirements,  some  have 
suggested  that  the  term  “common”  picture  may  be  a  misnomer  (R.  Munden,  personal 
communication,  November  23, 1996).  The  authors  do  not  concur:  a  force’s  duty  positions  and 
information  requirements  are  shaped  by  a  common  job  setting,  the  battlefield  situation,  and  a 
common  mission  task.  By  analogy,  the  notes  relevant  to  each  musician  depict  their  unique  portion 
of  the  “same  sheet  of  music.”  The  relevant  notes  or  symbols  for  each  player  are  choreographed  to  a 
common  score,  a  common  battle. 

This  report,  therefore,  recommends  methods  for  assessing  informational  relevance  based  on 
duty  position.  Alternative  recommendations,  not  pursued  in  this  report,  include  relevance  by 
echelon2  and  individual.  Ultimately,  C4I  displays  may  allow  individuals  to  tailor  the  situation 
depicted  to  match  their  unique  mental  model.  Filters  or  system  settings  that  automatically  gather 
relevant  information  and  gate  irrelevant  information  could  allow  individuals  to  tailor  or  customize 


2  For  a  more  detailed  discussion  of  viewpoint  differences  by  military  echelon  see  the  U.S.  General  Accounting  Office 
(1991). 
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a  battlefield  picture  to  their  own  area  of  interest,  a  user-defined  battlespace.  In  an  effort  to  bound 
the  definitional  problem,  however,  the  more  immediate  issue  pursued  in  this  report  is  information 
relevance,  by  duty  position,  on  a  C4I  display. 

Additionally,  the  notion  of  relevance  includes  timeliness.  Various  definitions  of  the 
common  picture  have  cited  a  requirement  for  real-time,  or  near-real-time,  battlefield  depictions. 
Military  practice  and  terminology,  such  as  latest  time  information  of  value  (LTIOV),  underscores 
the  temporal  aspect  of  informational  relevance.  To  develop  an  operational  definition  of  the 
common  picture  that  includes  timeliness,  the  currency  of  the  information  depicted  on  a  C4I  display, 
evaluation  methods  should  explicitly  address  temporal  requirements. 

Only  the  users  of  a  tool  can  adequately  define  the  tool’s  requirements.  Adequate  standards 
for  informational  content  and  relevance,  including  timeliness,  on  a  C4I  display  are  needed  to  define 
the  common  picture  requirement.  Adequate  research  methods  are  needed  to  help  users 
operationally  determine  their  common  picture  information  requirements. 

Determine  Information  Requirements 

This  section  begins  with  a  review  of  research  methods  for  determining  information 
requirements  called  knowledge  elicitation  methods.  The  methods  reviewed  provide  a  basis  for  the 
method  recommendations  that  immediately  follow.  These  recommendations  attempt  to  improve 
upon  the  methods  reviewed,  and  adapt  them  to  operationally  define  users'  common  picture 
requirements,  as  previously  bounded.  Method  recommendations  include  both  pragmatic  and 
thorough  approaches  to  knowledge  elicitation.  The  concluding  recommendation,  however,  is  a 
more  thorough  method  based  on  expert  observation  of  the  information  required  by  duty  position 
during  conduct  of  structured  training  exercises. 

Although  the  methods  recommended  rely  on  expert  performers  and  observers,  a  cognitive 
analysis  of  users’  information  requirements  should  include  both  experts  and  novices.  Display 
representations  appropriate  to  the  mental  models  of  experts  may  be  an  operational  objective  for  C4I 
systems.  Such  representations  may  also  be  needed  to  support  more  terminal  training  objectives. 
Initial  training  objectives,  however,  should  require  that  instructional  content  relates  to  novices' 
mental  models. 

If  the  goal  of  a  C4I  display  is  to  construct  an  external  reality  consistent  with  the  mental 
constructs  of  the  user,  research  tools  are  required  to  elicit  and  make  explicit  the  worker’s  mental 
model,  the  relevant  knowledge  base  of  the  user.  As  noted,  an  ISD-based  analysis  of  training 
content  or  tasks  addresses  the  external  structure  of  information  independent  of  its  internal 
representation.  In  contrast,  a  cognitive  analysis  should  help  identify  the  relevant  data  elements  and 
informational  patterns  of  a  user,  and  how  these  patterns  relate  to  external  structures  and 
performance  requirements  (Cooke,  1994). 

For  a  thorough  review  and  analysis  of  knowledge  elicitation  methods  see  Cooke  (1994). 
Cooke’s  review  summarizes  the  dimensions  and  tradeoffs  associated  with  three  families  of 
knowledge  elicitation  techniques:  observations  and  interviews;  process  tracing,  such  as  protocol 
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records  during  task  performance;  and  conceptual  techniques  directed,  for  example,  at  defining, 
analyzing,  and  interpreting  the  structure  of  a  knowledge  domain.  Cooke’s  review  also  provides 
general  recommendations  about  the  types  of  techniques  applicable  to  different  knowledge 
documentation  goals. 

Pertinent  examples  of  knowledge  elicitation  methods  used  to  identify  the  informational 
elements  and  battlefield  patterns  of  military  experts  during  mission  planning,  are  provided  by 
Fischer  &  Geiwitz  (1996).  Initially,  an  Information  Needs  Analysis  (INA)  method  was  used  to 
elicit  general  battlefield  patterns.  The  INA  method  assumed  the  required  information  was  in  the 
standard  stimulus  materials  provided,  such  as  operations  orders  and  plans,  for  the  mission  planning 
phase.  Experts  were  interviewed  to  identify  the  battlefield  patterns  they  detected  in  the  stimulus 
materials.  Next,  sorting  methods  required  other  experts  to  classify  operational  orders  and  78 
different  battlefield  elements,  such  as  mission,  resources,  task  organization,  and  enemy  intentions, 
into  meaningful  categories.  Finally,  interview  and  protocol  analysis  methods  were  used  to  elicit  40 
different  terrain  patterns  with  military  significance. 

Related  examples  of  knowledge  elicitation  methods  are  research  on  similar  battlefield 
situations  and  the  metacognitive  skills  for  battlefield  situation  assessment  by  Federico  (1995; 

1997).  Both  of  these  efforts  employed  similar  stimulus  materials  in  which  Navy  participants’ 
knowledge  of  the  stimulus  situations  was  initially  elicited  by  card-sorting  the  situations  into 
discrete  clusters  and  then  labeling  the  clusters.  The  work  on  similar  situations  addressed  expert 
versus  novice  knowledge  bases  by  requiring  participants  to  weigh  the  importance  of  informational 
elements  on  the  situation,  their  source,  and  their  similarity  (Federico,  1995).  The  work  on 
metacognitive  skills  for  situation  assessment  addressed  how  the  mental  models  of  military 
personnel  affect  their  perception  of,  and  decision-making  on,  battlefield  situations  (Federico,  1997). 

In  addition  to  card-sorting  and  labeling,  other  knowledge  elicitation  methods  used  by 
Federico  (1997)  included  pair-wise  similarity  ratings  of  stimulus  situations  (1-7  point  rating  scales), 
and  participants’  representations  of  their  process  models  for  tactical  decision-making.  Participants 
illustrated  their  own  model  for  decision-making  by  spatially  arranging  its  cognitive  components,  to 
include:  enemy  information,  scenario  event  sequence,  participant’s  formal  training  on  doctrine  and 
tactics,  participant’s  experience  and  observations  in  military  exercises,  situation  assessment, 
similarity  recognition,  and  tactical  action.  Elicitation  methods  required  participants  to  weigh  and 
link  these  components,  to  draw  directional  arrows  for  these  links,  and  to  specify  any  sequences  of 
activity  among  the  components.  Although  research  methods  are  the  primary  concern  of  this  report, 
the  research  findings  reinforced  the  effect  of  perceptual  processes  on  tactical  decisions  and  actions 
(Federico,  1997). 

The  end  result  of  knowledge  elicitation  methods  is  a  model  that  represents  the  referent 
object  with  varying  accuracy  (Cooke,  1994).  The  more  specified  or  formal  the  method  used  to 
elicit  knowledge,  the  more  systematic  and  complete  the  model  and  the  more  quantitative  the  results. 
Formal  methods  too  divorced  from  actual  task  performance  often  lack  validity,  however,  and 
require  extensive  preparation  time  and  domain  knowledge.  Less  formal  methods,  particularly  task 
based,  may  yield  valuable  knowledge  elements;  but  they  often  result  in  a  glut  of  qualitative 
information  that  is  hard  to  analyze  or  organize  into  a  coherent  model. 
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The  method  recommendations  on  knowledge  elicitation  for  defining  the  common  picture 
are  based  on  the  methods  reviewed.  The  recommended  adaptations  of  these  methods  address  some 
of  their  perceived  shortcomings  and  the  information  requirements  unique  to  the  common  picture 
depicted  on  C4I  displays.  These  methods  stress  a  more  formal  approach  to  knowledge  elicitation. 

In  particular,  they  stress  the  need  to  ground  the  elicitation  of  required  informational  elements  for  the 
common  picture  in  well-structured  military  scenarios.  These  scenarios  should  provide  microworld 
models  of  the  operational  context,  battlefield  situations  during  the  mission,  and  a  detailed 
representation  of  that  context  for  each  key  duty  position. 

A  perceived  weakness  in  the  methods  reviewed  was  a  reliance  on  relatively  impoverished 
stimulus  materials.  For  example,  Fischer  and  Geiwitz  (1996)  based  their  analysis  of  meaningful 
battlefield  patterns  only  on  planning-level  materials,  and  failed  to  document  the  source  or  quality  of 
materials  used.  The  types  of  battlefield  patterns  that  emerge  during  the  more  dynamic  and  stressful 
mission  phases,  to  include  mission  execution,  were  not  addressed.  Similarly,  Federico’s  work  was 
based  on  “seven  abbreviated  and  telegraphic  tactical  situations,  each  of  which  were  written  on  a  5 
1/2x8  1/2-inch  numbered  note  card”  (Federico,  1997,  p.  151).  While  the  materials  used  may  have 
been  adequate  for  their  purposes,  more  comprehensive  and  tested  source  materials  are  needed  to 
define  the  common  picture.  In  contrast,  the  FXXITP  training  exercises  provide  a  structured  set  of 
stimulus  materials.  The  FXXITP  training  exercises  are  recommended,  therefore,  for  both  the 
pragmatic  and  more  thorough  methods  recommended  in  the  following  sections. 

More  Pragmatic  Knowledge  Elicitation  Methods 

To  determine  user  information  requirements,  alternative  knowledge  elicitation  methods  are 
recommended  in  order  to  balance  effectiveness  and  efficiency  considerations.  Both  the  more 
pragmatic  and  thorough  methods  recommended,  however,  propose  that  knowledge  elicitation 
should  be  structured  by  the  factors  of  METT-T.  A  notional  example  of  some  coarse-grained 
informational  elements  for  each  METT-T  factor  was  provided  in  Table  2,  and  more  detailed 
informational  elements  would  be  expected  as  a  result  of  knowledge  elicitation.  A  pragmatic 
approach  to  eliciting  FXXITP-based  informational  requirements  might  rely  on  subject  matter 
experts  (SMEs),  such  as  the  dedicated  team  of  observers/controllers  who  routinely  administer 
FXXITP  training  exercises,  or  the  FXXITP  developmental  teams. 

Table  3  presents  a  notional  METT-T  checklist  to  structure  SME  knowledge  elicitation.  For 
a  sample  of  representative  duty  positions,  SMEs  could  determine  what  informational  elements  were 
required  for  each  aspect  of  METT-T  during  a  FXXITP  exercise  by  phase  and  event.  At  the  same 
time,  these  SMEs  might  estimate  the  relevance  and  criticality  of  each  informational  element  on  a 
multipoint  rating  scale,  with  respect  to  a  user’s  duty  position  and  to  overall  mission  success.  They 
might  also  estimate  when  the  informational  element  was  needed  and  when  it  was  no  longer  of 
value,  in  relation  to  the  FXXITP  timeline  of  battlefield  events.  In  addition,  these  SMEs  could 
provide  estimates  about  information  flow.  These  methods  should  identify  the  original  source  of  the 
information,  the  path  taken  to  get  the  information  from  the  source  to  the  duty  position  under 
examination,  and  any  paths  taken  to  distribute  the  information  to  others. 
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Table  3 


Notional  Checklist  for  Identifying  METT-T  Information  Requirements 


Mission/Phase/Event 


Reqd 

Info 

Relevant 

Info 

Critical 

Info 

Y/N 

1  2345 

1  2345 

□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


Time  Time 
Reqd  Recvd 


□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

□  □□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

Y/N 

□□□□□ 

□□□□□ 

□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


□  □□□□ 


Information  Information 

Source  Destination 


Dir  DP  Dir  DP 


ft 

ft 

■=> 

ft 

ft 

«=> 

ft 

ft 

ft 

ft 

ft 

ft 

tt 

ft 

ft 

ft 
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ft 

ft 

it 

ft 

it 

ft 
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ft 

ft 

ft 

ft 

ft 

ft 

ft 

ft 

O 
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ft 

ft 

ft 
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ft 

ft 

ft 

ft 

c> 
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□  □□□□ 


Note:  Info  =  Information;  Reqd  =  Required;  Recvd  =  Received;  Dir  = 
Unit);  DP  =  Duty  Position;  Comm  =  Communicate;  Aves  =  Avenues. 
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Direction  (Higher,  Lower,  or  Adjacent 
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To  maintain  a  pragmatic  approach,  these  SMEs  might  base  their  estimates  solely  on  the  off- 
the-shelf  stimulus  materials  in  a  FXXITP  training  support  package  (vs.  actual  observation  of  a 
FXXITP  exercise)  and  their  own  subject  matter  expertise.  The  FXXITP’s  detailed  exercise 
materials  should  serve  to  stimulate  the  recall  of  information  requirements  by  the  SMEs.  Their 
estimates  might  be  based  on  the  available  set  of  battalion  or  brigade  FXXITP  exercises,  or  more 
pragmatically  on  a  subset  of  exercises.  Similar  stimulated  recall  methods  might  be  used  to  elicit  the 
informational  requirements  of  users,  rather  than  SMEs,  after  the  users  have  completed  a  subject 
FXXITP  exercise. 

More  Thorough  Knowledge  Elicitation  Methods 

More  thorough  approaches  to  knowledge  elicitation  could  provide  a  firm  basis  for 
operationally  defining  common  picture  information  requirements,  and  supporting  future  training 
and  evaluation  efforts.  More  thorough  methods  would  extend  the  pragmatic  methods  described  to 
include  all  duty  positions  in  a  battalion  or  brigade  unit.  Additionally,  more  thorough  knowledge 
elicitation  methods  called  process  tracing  would  require  that  information  requirements  be  elicited 
during  actual  task-based  performance  (Cooke,  1994)3.  Process  tracing  to  elicit  the  knowledge  of 
skilled  performers  during  an  exercise  is  informative  and  potentially  more  credible  than  observer 
data,  but  it  may  disrupt  and  contaminate  the  subject  process.  To  elicit  knowledge  from 
nonperformers,  such  as  SMEs  observing  actual  or  prerecorded  task-based  performance,  process 
tracing  is  less  disruptive  and  allows  a  more  structured  and  directed  approach  to  knowledge 
elicitation. 

The  primary  method  recommendation,  therefore,  is  process  tracing  to  elicit  SME’s 
knowledge  of  METT-T  information  requirements  based  on  their  observations  of  FXXITP  exercises 
conducted  by  skilled  performers.  This  method  should  target  the  METT-T  aspects  deemed  relevant 
by  the  SME  for  each  assigned  duty  position.  The  SME  protocol  might  be  a  checklist  of  METT-T 
elements  and  information  exchange  requirements  similar  to  Table  3,  by  exercise  phase  and  event. 
The  information  elicited  could  supplement  that  gained  from  more  pragmatic  methods.  It  might 
better  identify  what  information  is  considered  by  experts,  the  order  in  which  the  information  is 
considered,  and  the  actual  rules  used  to  combine  information  (Cooke,  1994). 

In  summary,  the  knowledge  elicitation  methods  recommended  crosswalking  the 
information  requirements  of  skilled  performers  elicited  during  FXXITP  exercises  with  FBCB2 
processing  and  display  capabilities.  The  recommendation  to  include  all  the  key  duty  positions  in  a 
battalion  or  brigade  should  result  in  a  comprehensive  database  that  captures  the  informational 
requirements  of  the  entire  unit.  In  large  part,  this  database  operationally  defines  the  information 
required  to  develop  and  maintain  a  common  picture  of  the  battlefield,  within  the  bounded  problem 
space.  It  also  provides  an  empirical  base  to  train  and  assess  information  management  performance, 
such  as  the  unit's  ability  to  maintain  a  common  picture  of  their  battlefield  situation. 


3  Notably,  a  log  of  human-computer  interactions  compiled  during  actual  task  performance  is  a  form  of  process  tracing, 
although  not  knowledge  elicitation,  that  is  considered  later  in  this  report. 
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Assess  C4I  Information  Capabilities 


Perceived  shortcomings  in  the  Cl  systems  currently  available  to  the  Army  are  surfacing  the 
need  for  more  precise  specification  and  testing  of  developmental  C4I  systems.  Advocates  for  this 
need  urge  an  a  priori  approach  in  which  required  capabilities  are  validated,  for  instance  in  a  virtual 
simulation  environment,  prior  to  more  costly  and  frequently  failed  field  exercises  or  evaluations  (J. 
Hiller,  personal  communication,  February  18, 1997).  More  specifically,  research  on  Cl  systems 
should  develop  and  apply  “measures  of  effectiveness  (MOEs)  reflecting  the  performance  of  the 
hardware”  (J.  Hiller,  personal  communication,  December  9, 1996).  Notably,  the  Army's  current  Cl 
systems  may  not  be  capable  of  displaying  many  of  the  informational  elements  required  to  meet  the 
common  picture  objective.  For  example,  a  current  Cl  system  for  the  Abrams  tank,  the  Inter- 
Vehicular  Information  System  (IVIS),  displays  only  a  grid  matrix  for  the  geographic  area  depicted 
that  provides  no  terrain  features,  such  as  hills,  roads,  and  rivers. 

The  method  recommendation  to  address  this  research  issue  directly  assesses  Cl 
performance  against  user  information  requirements.  This  method  builds  on  the  information 
requirements  previously  determined  by  applying  the  methods  recommended  in  the  above  section. 
Once  user  information  requirements  are  elicited  and  codified  by  METT-T,  they  should  provide  an 
empirical  basis  for  assessing  the  extent  to  which  a  Cl  system  meets  those  requirements.  This 
report's  pictorial  focus  leads  to  a  method  recommendation  that  results  in  a  set  of  Cl  display  codes 
directed  at  the  common  picture  objective.  These  Cl  display  codes  would  equate  to  MOEs 
reflecting  machine  performance,  and  help  justify  required  performance  improvements. 

Cl  Display  Codes 

While  various  methods  are  applicable  to  assessing  machine  performance,  the  method 
recommend  is  an  adaptation  of  a  method  developed  by  Burnside  (1990)  for  assessing  the 
capabilities  of  training  simulations.  For  training  assessment,  Burnside’s  method  first  identified  the 
population  of  tasks  to  be  trained  in  a  given  domain,  battalion-down  tank  and  mechanized  infantry 
units.  Later,  the  method  determined  the  degree  to  which  the  standards  for  those  tasks  could  be  met 
in  simulation,  namely  the  Simulation  Networking  (SIMNET)  virtual  simulation.  For  Cl  system 
assessment,  the  method  would  begin  with  the  population  of  METT-T  informational  requirements 
for  a  battalion  or  brigade,  previously  identified  through  knowledge  elicitation  methods.  Later,  the 
method  would  determine  the  degree  to  which  the  informational  elements  could  be  displayed  by  a 
Cl  system,  such  as  FBCB2. 

A  recommended  method  for  deriving  Cl  display  codes  by  METT-T  would  require  SME 
ratings  for  informational  elements  within  each  factor.  Elements  of  required  Enemy  information,  for 
example,  might  include:  size,  activity,  location,  unit  type,  time,  equipment,  and  intent.  Ratings  at 
this  more  precise  level  should  result  in  more  reliable  estimates.  The  criteria  for  estimating  Cl 
display  capability  might  use  a  5-point  rating  scale  with  clearly  anchored  definitions  for  each  of  the 
five  categories  that  track  both  textual  and  graphic  display  formats.  Criteria  might  range,  for 
example,  from  “Not  Displayed”  to  “Fully  Displayed  Graphically.”  Raters  would  make  their 
estimates  independently  by  observing  each  of  the  informational  elements  as  displayed,  or  not,  on 
the  target  Cl  system,  such  as  FBCB2.  A  display  driver  might  be  used  to  automatically  present  the 
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elements,  as  ordered  on  the  rating  form,  to  expedite  and  standardize  stimulus  presentation  and 
procedure. 

Decision  rules  are  needed  to  consolidate  the  ratings  for  each  type  of  informational  element 
into  each  of  the  five  METT-T  categories,  and  in  turn,  into  an  overall  METT-T  assessment.  The 
decision  rules  formulated  by  Burnside  (1990)  for  consolidating  ratings  within  and  across  categories 
(subtask  and  task  categories,  for  Burnside’s  effort)  could  be  adapted  to  the  METT-T  ratings.  As 
noted  by  Burnside,  such  decision  rules  are  algorithmic  and  can  be  automated  to  avoid  manual 
calculation  of  the  required  estimates. 

After  these  rules  are  formulated  and  applied,  more  objective  and  meaningful  assessments  of 
the  common  picture  requirement  should  result.  A  notional  estimate  for  the  factor  Enemy,  for 
example,  might  be:  “The  FBCB2  application  graphically  displays  60%  of  the  enemy  information 
required  by  a  battalion  commander.”  Estimates  by  duty  position  should  help  assess  information 
requirements  for  any  METT-T  factor,  or  across  METT-T.  Estimates  across  different  duty  positions 
could  provide  a  basis  for  consolidating  information  requirements  by  echelon,  unit,  or  battlefield 
functional  area. 

Such  estimates  should  provide  a  basis  for  quantifying  and  categorizing  specific  C4I 
requirements  as  met  or  not  met.  At  lower-levels,  estimates  for  each  type  of  informational  element 
rated  would  provide  a  basis  for  more  precise  requirements  and,  in  turn,  more  exacting  system 
specifications  for  improvements  and  enhancements.  These  more  detailed  estimates  would  also 
provide  a  basis  to  identify  and  develop  digital  training  and  evaluation  methods,  as  discussed  later. 
More  global  estimates,  by  METT-T  category  or  across  METT-T,  might  provide  a  more  succinct 
and  defensible  assessment  of  C4I  display  capabilities.  Assessments  at  this  level  would  allow  the 
Army  to  objectively  report  status  on  the  common  picture  objective  and  justify  support  needed. 

Assess  Performance,  Give  Feedback 


Measurement  is  essential  to  training  and  evaluation.  However,  direct  observation  and 
objective  measurement  of  performance  during  military  training  exercises  is  difficult,  particularly 
for  command  and  control  performance.  Research  methods  are  needed  to  provide  an  empirical 
account  of  the  pictorial  elements  required  for  the  common  picture  product  and,  the  process  by 
which  those  elements  are  created  and  maintained.  Training  methods  are  needed  to  provide 
empirical  assessment  and  feedback  on  trainees'  performance  related  to  product  and  process 
requirements.  The  method  recommendation,  therefore,  is  to  instrument  C4I  systems  to  compile  a 
log  of  all  soldier-computer  interactions  as  an  empirical  basis  for  assessing  performance  and  giving 
feedback.  Before  discussing  this  recommendation,  the  distinction  between  physical  and  mental 
“pictures”  of  the  battlefield  situation  is  considered. 

This  report's  emphasis  on  empirical  methods  is  based  on  the  pictorial  elements  depicted  on 
a  C4I  display.  These  pictorial  elements  on  a  C4I  display  constitute  a  physical  and  objective  picture 
of  the  battlefield  situation,  albeit  not  necessarily  an  accurate  one.  Soldiers  and  their  C4I  systems 
determine  what  pictorial  elements  are,  and  are  not,  depicted.  Each  and  every  pictorial  element  on  a 
tactical  display  can  be  empirically  traced  to  its  source,  soldier  or  machine,  including  the  path  from 
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that  source  to  the  user  of  the  display.  The  method  recommendation  will  examine  how  this 
empirical  data  can  be  automatically  compiled  and  assessed  to  provide  performance  feedback. 

In  contrast,  the  term  common  picture  is  frequently  used  to  describe  the  user’s  mental  picture 
of  a  battlefield  situation  or  “situational  awareness”  (DA,  1996b).  The  distinction  between  physical 
and  mental  pictures  seems  blurred,  for  example,  by  an  Army  definition  of  situational  awareness. 
“Situational  awareness  is  the  ability  to  have  accurate  and  real-time  information  of  friendly,  enemy, 
neutral,  and  noncombatant  locations;  a  common,  relevant  picture  of  the  battlefield  scaled  to  specific 
level  of  interest  and  special  needs”  (DA,  1998a,  p.  5).  Similarly,  the  Army’s  training  requirement 
for  situational  awareness  is  closely  related  to  the  user’s  digital  display  (DA,  1997a): 

Capability  to  create  an  accurate  and  high  fidelity  ...collaborative  real-time  picture  of  the 
battlespace  to  include  weather,  terrain,  environment,  and  friendly/enemy/neutral/non- 
combatant  situational  and  status  information ....  The  common  picture  provides 
understanding  of  available  information  in  terms  of  the  battlespace:  width,  depth,  height, 
position,  time,  terrain,  materiel,  weather,  obstacles  and  barriers ....  The  relevant  common 
picture  must  be  scaleable  to  appropriate  levels  of  command,  tailorable  by  function  and 
personal  preference,  and  based  on  variable  user  defined  parameters....  (pp.  5-6) 


Mental  pictures  of  a  situation  or  situational  awareness,  however,  are  subjective  and  difficult 
to  measure  (Adams,  Tenney,  &  Pew,  1995;  Endsley,  1995).  They  are  influenced  by  individual 
differences  such  as  expertise  and  motivation,  and  ideally  by  the  physical  picture  on  a  C4I  display. 

A  telling  distinction,  the  same  picture  on  a  C4I  display  may  result  in  very  different  mental  pictures 
by  different  users,  such  as  an  expert  versus  a  novice.  In  sum,  the  distinction  is  fundamental  to  this 
report's  emphasis  on  the  common  picture  as  an  empirical  construct  based  solely  on  the  physical 
pictures,  the  pictorial  elements,  on  a  C4I  display. 

Instrument  C4I  Systems 


Digital  C4I  systems  are  uniquely  suited  to  automatically  collect  data  for  performance 
assessment  and  feedback.  Computers  can,  and  frequently  do,  routinely  log  all  user  inputs  and 
system  responses.  For  this  report,  instrumentation  of  a  C4I  system  equates  to  a  system  log  of  all 
soldier-computer  interactions  correlated  with  the  simulated  battlefield  situation  in  which  they  occur. 
No  currently  fielded  C4I  systems  are  instrumented,  however.  The  method  recommendation, 
therefore,  is  to  instrument  C4I  systems.  This  recommendation  includes  all  C4I  system  variants — 
operational,  developmental,  and  training. 

Method  recommendations  for  applying  instrumented  C4I  systems  begin  with  a  discussion  of 
the  common  picture  as  a  meaningful,  measurable,  and  collaborative  product.  This  discussion 
describes  how  instrumented  C4I  systems  could  automatically  capture  this  product  at  any  moment 
during  a  training  exercise  as  well  as  the  collaborative  process  by  which  this  product  is  created  and 
maintained.  Next,  methods  for  assessing  performance  and  providing  feedback  based  on 
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instrumented  data  are  considered.  Finally,  a  working  example  of  this  method  is  described  called 
automated  pictorial  comparisons,  based  on  normative  or  criterion  standards  of  a  common  picture. 

A  Meaningful,  Measurable,  and  Collaborative  Product.  The  physical  picture  depicted  on  a 
C4I  tactical  display  equates  to  a  meaningful,  measurable,  and  collaborative  product.  A  display 
depiction  is  meaningful  if  it  provides  a  representation  of  the  external  world  that  perceptually  maps 
to  the  mental  model  of  the  viewer.  A  display  depiction  is  measurable  if  it  provides  access  to  its 
underlying  database.  Key  data  for  assessing  a  computer-based  tactical  display  includes  the  data 
elements  depicted  on  the  display  and  their  collaborative  source.  Source  data  includes  a  record  of  all 
soldier-computer  interactions  required  to  generate  the  user’s  display,  or  intra-system  data.  Source 
data  also  includes  all  information  from  external  sources  such  as  messages  received  from  other  users 
and  the  path  to  that  source,  or  inter-system  data.  A  composite  database  of  intersystem  and 
intrasystem  data  is  readily  obtained  if  C4I  systems  are  instrumented  to  provide,  in  essence,  a  log  of 
all  soldier-computer  interactions  (Lickteig,  1996). 

Examples  of  intersystem  data  include  the  receipt  and  content  of  mission  orders,  graphic 
overlays,  and  recurrent  updates  on  enemy  and  friendly  locations  and  identities.  They  also  include 
requests  for  information  and  for  support,  such  as  indirect  fires,  close  air,  and  supplies.  Intersystem 
data  also  includes  what  information  was  distributed  by  each  C4I  user  as  intersystem 
communications  to  others.  Such  data  disclose  what  battlefield  information  was  available  and  not 
available  to  C4I  equipped  soldiers,  and  when  that  information  was  available. 

Examples  of  intrasystem  data  include  when  and  if  intersystem  data  received  by  a  user’s  C4I 
system,  and  therefore  available,  was  “opened”  for  presentation  on  the  user’s  display.  They  also 
include  when  and  if  that  information  was  actually  visible  in  the  user’s  display  “window”  based  on 
the  map  areas  and  scales  selected  by  the  user.  Intrasystem  data  also  disclose  what  information  was 
solicited  and  examined  by  the  user  based  on  a  log  of  user  interactions,  such  as  the  activation  of 
terrain  analysis  tools  or  the  call  up  of  friendly  status  menus. 

This  composite  database  of  the  elements  depicted  on  the  tactical  display  by  source  and  path 
represents  a  measurable  product  achieved  through  collaborative  information  processing  and 
management.  This  collaboration  includes  all  individual  and  collective  soldier-computer 
interactions  required  to  create  and  maintain  the  battlefield  pictures  depicted  on  a  unit's  C4I  displays. 
In  addition,  this  database  provides  a  quantifiable  and  tractable  link  between  the  informational 
requirements  of  the  users  and  the  informational  capabilities  of  the  tactical  display.  Notably,  this 
composite  database  is  used  to  support  the  training  methods  recommended  later  in  this  report. 
Intrasystem  data  would  support  individual  training  analysis  and  feedback  methods.  Inter-  and  intra¬ 
system  data  would  support  small  group  and  collective  training  analysis  and  feedback  methods. 

A  Working  Example:  Automated  Pictorial  Comparisons.  A  working  example  of 
collaborative  products  based  on  this  composite  database  is  automated  pictorial  comparisons  of  the 
battlefield  situations  depicted  on  users'  C4I  displays  at  any  time  during  a  simulated  mission 
exercise.  Such  comparisons  are  analogous  to  the  "compare  document"  function  provided  by  many 
word  processing  applications  for  comparing  textual  products.  For  most  text  applications,  this 
function  automatically  compares  multiple  versions  of  a  product  and  highlights  detected 
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discrepancies  for  both  on  screen  and  hard  copy  presentation.  Users  are  often  provided  highlight 
options,  such  as  color,  bold,  underline  and  strikethrough,  to  accent  discrepancies  and  identify  the 
source  or  author. 

Figure  1  provides  a  simple  example  of  how  automated  pictorial  comparison  might  illustrate 
a  key  discrepancy  between  the  battlefield  representations  depicted  on  the  C4I  displays  of  two 
different  users  at  a  selected  moment  during  a  mission.  In  this  figure  example,  a  company 
commander’s  display  at  1200  hr  depicts  four  Enemy  tanks,  a  platoon  sized  unit,  that  is  not 
simultaneously  depicted  on  another  display  at  the  next  level  of  command,  the  battalion  commander. 
Such  a  potentially  important  discrepancy  would  be  automatically  detected  and  highlighted  by  the 
proposed  compare  picture  routine.  Instrumented  C4I  systems  could  readily  extend  this  example  to 
compare  the  battlefield  situations  depicted  on  the  C4I  displays  of  each  user  at  any  time  during  a 
simulated  mission  exercise. 


Company  Commander’s  Display 


Battalion  Commander’s  Display 


Enemy  Platoon  NW  at  1200  hr 


No  Enemy  Platoon  NW  at  1200  hr 


Figure  1 .  An  example  of  automated  pictorial  comparisons  on  notional  C4I  displays. 

This  report's  method  recommendation  for  automated  pictorial  comparisons  underscores  the 
need  for,  and  potential  of,  instrumented  C4I  systems.  A  comprehensive  log  of  soldier-computer 
interactions  correlated  with  the  simulated  battlefield  situation  in  which  they  occur  would  provide 
an  unprecedented  empirical  base  for  training  and  evaluation.  More  definitive  and  meaningful 
comparisons,  however,  require  the  logging  of  intrasystem  data  to  accurately  assess  actual  display 
content.  In  contrast,  pictorial  comparisons  based  only  on  intersystem  data  are  speculative.  For 
example,  work  by  Brown,  Metzler,  Riede  and  Wonsewitz  (1996)  provide  excellent  examples  of 
pictorial  comparisons  for  training  feedback  based  on  intersystem  data.  Such  intersystem 
comparisons,  however,  disclose  only  the  information  that  should  have  been  available  to  a  user, 
rather  than  the  information  that  was  actually  visible  to  each  user  at  any  moment  during  a  simulated 
battlefield  exercise. 


Pictorial  comparisons  are  not  limited  to  product  comparisons,  such  as  momentary  snapshots 
of  an  exercise.  More  process  directed  comparisons  might  illustrate  the  flow  of  information  across 
the  unit,  such  as  when  and  how  the  company  commander  received  enemy  information  that  was  not 
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received  by  the  platoon  leader.  A  different  type  of  process  comparison  might  highlight  intrasystem 
discrepancies  such  as  the  information  received  by  a  user's  C4I  system  that  was  never  opened  or 
actually  visible  in  the  user’s  C4I  display  window.  This  comparison  requires  only  an  automated 
analysis  of  the  map-based  location  of  the  information  depicted  relative  to  the  map  areas  actually 
displayed  to  the  user. 

Notably,  pictorial  comparisons  might  be  based  on  normative  or  criterion  standards.  One 
method  for  normative  comparisons  might  simply  compare  C4I  pictures  across  a  unit  or  any 
designated  set  of  unit  members,  such  as  the  battalion  commander  and  the  company  commanders. 
Such  comparisons  would  identify  and  highlight  what  is  uncommon  about  a  unit's  common  picture 
of  the  battlefield.  Although  some  differences  between  duty  positions  are  to  be  expected,  as 
previously  discussed,  the  comparisons  can  be  set  to  address  only  selected  types  of  commonly 
required  information.  One  type  might  compare  enemy  vehicles  depicted,  or  not  depicted,  on  unit 
members'  displays  at  any  time  during  the  exercise.  Other  types  of  normative  comparisons  within  a 
unit  might  include  any  other  METT-T  factor  or  informational  element  listed  in  Table  2. 

Criterion  standards  for  assessing  task  performance  related  to  the  common  picture  would 
provide  another  basis  for  pictorial  comparisons.  Such  comparisons  require  an  accepted  criterion  for 
what  should  be  depicted  on  a  C4I  display.  The  methods  previously  recommended  for  determining 
information  requirements  through  expert  knowledge  elicitation  were  designed  to  help  establish  such 
standards.  This  expert  analysis  included  consideration  of  the  corresponding  C4I  pictorial  elements 
associated  with  the  information  required.  One  goal  of  that  effort  was  to  help  develop  a  model  of 
required  information  exchanges  by  duty  position  for  battalion  or  brigade  unit  training.  Later  in  this 
report,  method  recommendations  for  developing  and  applying  a  computer-driven  version  of  this 
information  exchange  model  to  performance  assessment  and  feedback  are  presented.  Given  this 
model,  criterion-based  pictorial  comparisons  would  match  trainees'  product  and  process 
performance  with  corresponding  standards  from  the  information  exchange  model. 

The  remainder  of  this  report  will  examine  how  to  apply  automated  pictorial 
comparisons,  and  additional  examples  of  instrumented  C4I  measures,  to  efficiently  and  effectively 
assess  information  related  task  performance  and  provide  feedback  at  individual,  small  group  and 
collective  levels. 


Communicate  the  Common  Picture 

This  section  identifies  two  key  research  issues  to  improve  the  ability  of  soldiers  and  C4I 
systems  to  pictorially  communicate  battlefield  situations.  First,  the  soldier’s  mental  model  of  the 
battlefield  environment  should  shape  the  tool’s  microworld  representation  of  this  environment  to 
render  it  more  meaningful.  This  issue  is  identified  as  the  need  to  shape  C4I  representations. 
Second,  the  soldier’s  tool  or  C4I  system  should  shape  or  train  the  worker  to  recognize  and 
understand  the  tactical  patterns  and  situations  depicted  on  C4I  displays.  This  issue  is  identified  as 
the  need  to  shape  mental  models,  and  is  regarded  in  this  report  as  an  individual  training  issue. 
Method  recommendations  to  address  these  issues  are  based  on  the  literature  reviewed,  and  more 
detailed  method  descriptions  are  provided  and  referenced  to  clarify  the  method  recommendations 
and  support  their  application. 


33 


The  process  of  forming  meaningful  computer  representations  is  communication  between 
users  and  computers.  Method  recommendations  to  improve  soldier-computer  communication  will 
stress  the  interaction,  the  dialog,  between  human  and  computer  representations.  A  common 
example  of  how  a  user's  model  shapes  a  computer's  representation  is  a  backyard,  deck-design 
program  at  the  local  hardware  store.  Examples  of  how  the  computer  shapes  internal  models  are  as 
common  as  messages  received  from  e-mail  or  a  C4I  system.  A  less  common,  and  more 
pedagogical,  example  might  be  a  tactical  display  that  effectively  communicates  terrain  relief  by 
allowing  the  trainee  to  iteratively  transform,  as  desired,  a  2-dimensional  (D)  topographic  map  with 
contour  lines  into  a  3-D  perspective  view  of  terrain  elevation.  The  digital  training  methods 
recommended  provide  trainees  and  trainers  the  ability  to  manipulate  time  and  space  in  the  4-D 
battlefields  created  by  digital  technologies,  such  as  C4I  systems  and  warfighting  simulations. 

Method  recommendations  to  shape  C4I  representations  begin  by  assessing  how  well  C4I 
systems  pictorially  communicate  meaningful  information  to  the  user.  Then,  research  methods  are 
recommended  to  assess  the  tradeoffs  between  the  Army's  analog  information  formats  versus  current 
and  future  digital  formats.  Method  recommendations  to  shape  users'  mental  models  to  match  C4I 
representations  rely  on  documented  methods  for  developing  automatic  processing  capabilities  to 
train  pattern  and  situation  recognition  skills.  In  addition,  these  recommendations  capitalize  on  the 
ability  of  digital  technologies  to  provide  trainee  and  trainer  control  over  the  tactical  patterns  and 
battlefield  situations  depicted  on  C4I  displays.  A  variety  of  methods  to  control  or  manipulate 
training  presentation  formats  are  recommended  including:  a  reduced  stimulus  environment, 
animation,  perceptual  augmentation,  and  time  compression  and  expansion. 

Shape  C4I  Representations 

The  importance  of  perceptually  mapping  the  worker’s  mental  model  and  the  microworld 
work  model  was  previously  considered.  Unfortunately,  numerous  failures  in  commercial  computer 
systems  are  attributed  to  their  indifference  to  human  information  processing  and  problem  solving. 
In  contrast,  a  cognitively  engineered  interface  is  superior  in  performance,  satisfaction,  and 
workload  to  interfaces  that  are  not  cognitively  engineered  (Gerhardt-Powals,  1996).  One 
compelling  example  of  a  communicative  representation  is  a  direct-manipulation  interface,  such  as  a 
desktop  or  battlefield  metaphor.  By  depicting  direct  links  to  supporting  databases,  a  direct- 
manipulation  interface  aids  information  access,  extraction,  and  distribution;  by  depicting  a  coherent 
conceptual  schema,  it  aids  information  detection,  abstraction,  and  integration  (Lucarella  &  Zanzi, 
1996). 


As  a  metaphoric  battlefield,  the  common  picture  on  a  C4I  display  should  provide  a 
microworld  battlefield  representation  with  METT-T  object  domains.  This  representation  should 
afford  direct  manipulation  of  these  METT-T  objects,  at  the  level  the  user  chooses  to  transfer  goal¬ 
relevant  information  (Meshkati,  1996).  Advantages  of  such  a  representation  include  a  stable 
structure  that  naturally  nests  required  information  and  process  elements  (Bennett,  Nagy,  &  Flach, 
1997).  The  metaphoric  battlefield  would  also  readily  support  the  use  of  graphic  landmarks  to  aid 
navigation  among  dispersed  work  “areas”  while  maintaining  orientation.  Useful  landmarks  might 
include  a  picture-in-a-picture  overview  of  the  entire  battlefield  area  relative  to  the  area  currently 
displayed  in  the  interface  window,  or  depiction  of  a  menu  hierarchy  relative  to  current  location  on  a 
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multipage  menu.  For  a  recent  review  of  methods  that  support  user  configuration  of  their  work 
representations,  see  Pejtersen  and  Rasmussen  (1997). 

The  initial  concern  is  to  provide  a  method  that  allows  the  user  to  shape  the  microworld 
representation  of  the  battlefield  environment  on  a  C4I  display  to  render  it  more  meaningful.  In 
general,  the  cognitive  engineering  principles  and  guidelines  for  interface  design  and  training  were 
derived  from  a  useful  set  of  such  methods  (Gerhardt-Powals,  1996;  Lucarella  &  Zanzi,  1996).  In 
particular,  the  following  section  describes  a  further  modification  of  the  method  developed  by 
Burnside  (1990)  and  adapted  to  the  C4I  display  codes,  previously  discussed.  This  method  should 
result  in  Cl  display  decodes  based  on  the  user’s  ability  to  understand  and  use  the  information 
displayed. 

C4I  Display  Decodes 

This  method  recommendation  begins  with  the  set  of  METT-T  informational  elements 
previously  identified  as  Cl  display  codes  for  a  target  digital  system,  such  as  FBCB2.  This  method 
will  conclude  with  SME  or  user  estimates  of  how  understandable  and  useful  they  find  each  METT- 
T  informational  element,  as  currently  displayed.  These  decode  ratings  should  identify  perceived 
strengths  and  shortcomings  in  Cl  display  representation.  These  results  would  also  provide  a  basis 
for  comparing  current  Cl  display  formats  with  traditional  analog  formats  (generally  paper-based), 
and  alternative  digital  display  formats. 

Methods  for  deriving  Cl  display  decodes  would  again  require  SME  or  user  ratings  at  the 
level  of  discrete  informational  elements  within  each  METT-T  category.  Criteria  for  estimating  Cl 
display  understandability  and  usability  might  use  separate  5-point  rating  scales  with  clearly 
anchored  definitions  for  each  point.  For  understandability,  criteria  might  range  from  “Not 
Understood”  to  “Fully  Understood.”  For  usability,  criteria  might  range  from  “Not  Usable”  to 
“Fully  Usable.”  Raters  would  make  their  estimates  independently  by  observing  each  of  the 
informational  elements  as  displayed  on  the  target  Cl  system,  such  as  FBCB2.  The  rating  form 
should  request  comments  on  lower  rated  elements.  A  display  driver  that  automatically  presents  the 
elements,  as  ordered  on  the  rating  form,  might  again  expedite  and  standardize  stimulus  presentation 
and  procedure. 

Decision  rules,  similar  to  those  developed  for  Cl  display  codes,  could  consolidate  ratings 
by  informational  element  into  each  of  the  five  METT-T  categories  and  an  overall  METT-T 
assessment.  After  these  rules  are  formulated  and  applied,  estimates  of  understandability  and 
usability  could  be  compiled  across  rated  duty  positions  and  information  levels,  as  required. 

Overall,  such  estimates  might  identify  global  and  local  problems  in  understanding  and  acting  on  Cl 
battlefield  representations. 

Alternative  Format  Comparisons 


Research  methods  are  also  required  to  help  users  shape  the  microworld  representations 
displayed  by  Cl  systems.  Empirical  assessment  of  the  tradeoffs  associated  with  alternative  display 
formats  might  substantiate  a  requirement  for  format  revision  and  its  benefit.  For  such  comparisons. 
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the  methods  recommended  are  task-based  performance  evaluation  based  on  measures  of  speed, 
accuracy,  and  degree  of  understanding  and  usability.  The  number  and  type  of  required  comparisons 
would  depend  on  the  degree  of  decoding  deficiencies  previously  identified. 

Conventional  formats  for  military  information  rely  heavily  on  alphanumeric  characters, 
including  free-text  and  tabular  formats  (Gerhardt-Powals,  1 996).  In  addition  to  alphanumerics,  the 
military  also  uses  prescribed  graphical  presentation  formats  to  pictorially  depict  key  METT-T 
aspects  of  a  battlefield  situation  (DA,  1996d).  These  formats  include  a  standard  set  of  control 
measures  to  provide  a  visual  blueprint  of  the  mission,  and  a  standard  set  of  icons  symbolizing 
various  weapon  platforms  to  differentiate  unit  type,  size,  and  enemy  versus  friendly  alignment. 

They  also  include  symbols  and  legends  illustrating  key  terrain  features  and  patterns  of  tactical 
significance,  and  conventions  for  timelines  and  temporal  events  (e.g.,  proposed  future  locations). 

Currently,  most  C4I  systems  and  displays  have  adopted  the  standardized  text  and  graphic 
formats  used  by  the  military.  It  remains  an  empirical  question,  however,  how  understandable  and 
usable  such  symbology  is  when  converted  to  electronic  display  formats.  A  straightforward  set  of 
empirical  comparisons  to  identify  any  C4I  decoding  deficiencies  relative  to  the  military’s  standard 
presentation  formats  is  recommend  (DA,  1996d).  Given  the  display  limitations  of  current  C4I 
systems,  such  as  IVIS  and  FBCB2,  such  comparisons  may  disclose  that  paper-based  formats  are 
more  understandable  and  useful,  at  least  for  some  text  and  graphic  items.  A  more  realistic  context 
for  digital  formats  should  assess  information  decoding  and  use  on  a  cluttered  tactical  display  in 
which  multiple  items  are  concurrently  displayed. 

A  related  empirical  issue  is  whether  digital  technologies  might  provide  better  information 
formats  than  conventional  military  formats  and  symbols.  While  table  and  matrix  presentation 
formats  afford  precise  and  highly  detailed  information,  such  formats  may  not  provide  an  optimal 
representation  for  understanding  and  using  the  information  presented.  Tabled  information  is  often 
decontextualized  from  relevant  aspects  of  the  task  environment,  such  as  the  geometry  of  the 
battlefield.  For  example,  a  table  format  called  an  execution  matrix  is  used  to  coordinate  unit 
activities.  A  simple  example  of  an  execution  matrix  might  be  a  5  x  5  table  that  prespecifies  for 
each  of  five  units,  their  expected  locations  and  activities  for  five  successive  mission  segments.  The 
inherent  stability  of  alphanumeric  formats  (Kozma,  1991),  however,  may  not  readily  convey  the 
dynamic  patterns  implicit  in  an  execution  matrix. 

An  empirical  issue,  therefore,  is  to  determine  how  the  unique  information  processing  power 
of  digital  technologies  might  provide  alternative  formats  to  standard  military  symbology  to  increase 
user  understanding  and  use  of  the  information  displayed.  An  alternative  digital  format  for  an 
execution  matrix,  for  example,  might  be  an  animated  projection  of  successive  unit  locations  and 
activities  on  a  user’s  C4I  display  (Leibrecht,  Meade,  Schmidt,  Doherty,  &  Lickteig,  1994). 
Compared  to  a  tabular  format,  such  an  animation  might  significantly  improve  users’  speed, 
accuracy,  and  degree  of  understanding  and  usability. 

Additional  examples  of  alternative  digital  formats  will  be  considered  in  subsequent  sections 
that  attempt  to  apply  digital  technologies  to  the  common  picture  objective.  The  point  here  is  that 
empirical  comparison  might  identify  and  substantiate  the  requirement  for  alternative  digital 
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formats.  Such  comparisons,  based  on  SME  ratings  and  user  performance,  are  methods  for  shaping 
the  microworld  representation  of  the  battlefield  to  the  worker’s  mental  model  of  the  task 
environment. 


Shape  Mental  Models 

This  report's  application  of  digital  technologies  to  individual  training  begins  here  with 
issues  related  to  visual  recognition  of  the  battlefield  representation  depicted  by  C4I  displays.  The 
methods  recommended  in  this  section  stress  that  information  processing  and  display  capabilities  of 
the  Army’s  C4I  systems  should  shape  or  train  the  worker  to  better  understand  and  use  the 
information  and  representations  displayed.  For  enhancing  visual  recognition  of  the  battlefield 
representations  depicted  by  C4I  displays,  this  section  reviews  training  methods  from  two  distinct 
areas.  First,  training  methods  that  promote  an  automatic  recognition  of  patterns  and  situations  may 
provide  novices  many  of  the  component  skills  underlying  expert  performance  (Fisk  &  Rogers, 
1992).  Second,  digital  training  technologies  that  enable  trainees  and  trainers  to  control  or  transform 
microworld  representations  may  provide  unique  solutions  to  training  pattern  and  situation 
recognition  (Kozma,  1991). 

Automatic  processing  theory  assumes  that  training  entails  building  and  expanding  a  basic 
bank  of  component  skills  that  become  networked  and  organized  over  time  in  relation  to  task  goals 
(Fisk  &  Eboch,  1989).  Lower  level  or  component  skills  combine  with  others  to  form  a  component 
at  a  higher  level.  More  consistent  task  elements,  such  as  steering  or  braking  a  vehicle,  result  in 
more  automatic  processing  for  task  performance.  More  inconsistent  and  novel  task  elements,  such 
as  navigating  in  am  unfamiliar  city  or  terrain,  require  more  controlled  or  deliberate  processing  for 
task  performance.  For  the  Terrain  factor  of  METT-T,  for  example,  lower  level  component  skills 
may  equate  to  recognition  of  single-feature  patterns,  such  as  a  hill  or  slope.  Higher  level  skills  may 
equate  to  recognition  of  multiple-feature  patterns,  such  as  an  avenue  of  approach  or  road  network. 
Progressive  automaticity  should  network  and  organize  recognition  skills  across  both  of  these  levels, 
and  also  support  situation  recognition  skills  at  a  higher  level. 

Improvements  in  skilled  performance  are  primarily  due  to  developing  automatic  processes 
across  a  bank  of  supporting  component  skills  (Fisk  &  Eboch,  1989).  In  contrast,  controlled 
processes  show  little  improvement  with  practice,  and  controlling  resources  are  limited.  Some 
benefits  of  automated  processing  are: 

...attentional  requirements  are  minimized;  performance  becomes  faster  and  more  accurate; 

performance  becomes  less  susceptible  to  situational  stress;  and  performance  becomes  more 

durable  or  less  susceptible  to  memory  loss.  (Rogers,  Maurer,  Salas,  and  Fisk  1997,  p.  22) 

Although  the  benefits  of  habit  have  long  been  appreciated,  training  programs  are  rarely 
structured  to  fully  leverage  these  benefits.  For  example,  a  key  training  fallacy  is  that  practice 
makes  perfect  (Schneider,  1986).  A  poorly  structured  regimen  of  practice  often  results  in  little  or 
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no  improvement  in  performance.  Methods  for  training  automatic  processing  provide  explicit 
guidelines  on  how  to  structure  training  events  for  the  “consistent  practice”  required  to  improve 
performance  (Schneider,  1986). 

Second,  the  unique  capabilities  of  digital  technologies  to  represent  and  link  internal  and 
external  realities  afford  powerful  solutions  for  shaping  the  worker’s  mental  model  to  match  actual 
job  requirements  and  settings  (Fletcher,  1994;  Rasmussen  &  Pejtersen,  1995).  The  work  of  Kozma 
(1991, 1994)  includes  training  methods  that  allow  trainees  to  visualize  and  transform  microworld 
representations.  One  goal  of  such  methods  is  to  enable  novices  to  visibly  “see”  what  only  experts 
can  visualize.  For  example,  when  physicists  are  confronted  with  problem  situations  they  perceive 
patterns  based  on  underlying  structures  that  may  have  no  direct,  concrete  referent  in  the  real  world. 
Microworld  representations  of  the  same  problem  space,  however,  have  allowed  trainees  to  visualize 
and  manipulate  constructs  such  as  force  and  motion  directly.  Training  based  on  such  a  microworld 
representation  results  in  improved  understanding  and  use,  relative  to  more  traditional  training 
methods  (Kozma,  1994). 

Notably,  there  are  shortcomings  and  unintended  consequences  with  a  reliance  on  digital 
technologies  that  training  methods  must  avoid,  as  previously  discussed.  For  example,  there  is 
spreading  concern  that  soldiers'  progressive  dependence  on  digital  technologies  may  deter  the 
acquisition  and  maintenance  of  conventional  soldier  skills,  such  as  map  reading  (Ford,  Campbell,  & 
Cobb,  1998).  It  is  a  valid  concern,  but  one  that  does  not  fully  reflect  the  training  potential  of  digital 
technologies.  The  method  recommendations  stress  that  digital  technologies  are  an  exemplary 
medium  for  acquiring  and  maintaining  digital  and  conventional  skills,  such  as  pattern  and  situation 
recognition.  Soldiers  on  the  battlefield  must  possess  autonomous  skills  to  overcome  inevitable 
degradations  in  equipment,  such  as  C4I  systems.  The  Army  must  await  the  fielding  of  more 
advanced  and  fully  compatible  C4I  systems.  However,  it  can  leverage  the  capabilities  of  digital 
technologies  now  to  train  both  digital  and  conventional  skills,  such  as  pattern  and  situation 
recognition. 

Finally,  the  potential  realism  of  C4I  display  training  seems  underscored  by  emerging  trends 
in  information  age  warfighting,  such  as  increased  stand-off  ranges  and  indirect  vision  systems. 
Electronic  displays  and  their  microworld  representations  may  be  the  dominant  visual  channel  in  the 
beyond-line-of-sight  warfare  envisioned  for  the  future  battlefield. 

Pattern  Recognition 


This  section  begins  by  reviewing  training  methods  for  pattern  recognition  based  on 
automatic  processing  research,  and  then  recommending  how  to  apply  these  methods  to  digital 
technologies  to  train  the  recognition  of  tactical  patterns.  Next,  this  section  examines  how  digital 
technologies  can  allow  trainees  and  trainers  to  control  or  transform  patterns,  and  how  this  capability 
might  be  used  to  train  the  recognition  of  tactical  patterns.  While  the  primary  focus  here  is  the 
recognition  of  tactical  battlefield  patterns  as  represented  on  C4I  displays,  any  acquired  skills  should 
generalize  to  other  representations,  such  as  paper  maps,  and  actual  battlefield  situations. 
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Automatic  Processing.  Soldiers  must  be  able  to  recognize  the  tactical  battlefield  patterns 
depicted  on  their  C4I  displays  to  use  that  information  appropriately.  Skilled  pattern  recognition 
characterizes  expert  performance,  yet  the  Army  provides  no  formal  training  on  recognizing  and 
using  the  tactical  patterns  that  appear  on  the  battlefield  (Fischer  &  Geiwitz,  1996).  In  support  of  the 
common  picture  objective,  digital  technologies  provide  an  exemplary  medium  for  developing 
pattern  recognition  skills.  Recommended  methods  for  applying  digital  technologies  to  this  issue 
are  based  on  modifications  of  paper-based  methods  developed  by  Fischer  and  Geiwitz  (1996)  for 
pattern  recognition  training,  and  the  methods  used  by  Fisk  and  Eboch  (1989)  to  develop  automatic 
processing  of  map  reading  skills  related  to  map  legends. 

The  tactical  pattern  recognition  training  of  Fischer  and  Geiwitz  (1996)  stressed  extensive 
practice  with  feedback,  information  about  the  meaning  of  patterns,  the  perceptual  aspects  of 
patterns,  and  prototypical  patterns.  The  student  text  provided  prototype  or  exemplar  schematics  of 
each  terrain  pattern  accompanied  by  a  definition  and  a  description  of  the  critical  features  unique  to 
that  pattern.  The  approximately  40  hours  of  training  began  with  single-feature  patterns  such  as  a 
hill  or  ridge,  and  progressed  to  multiple-feature  patterns  such  as  canalizing  terrain  or  an  avenue  of 
approach.  Trainees  then  completed  workbook  exercises  that  required  them  to  identify  terrain 
features  on  military  topographic  maps,  and  mark  patterns  of  tactical  significance  within  designated 
map  areas,  such  as  unit  boundaries. 

Four  separate  measures  immediately  after  training,  and  a  retest  nine  months  later  were  used 
to  assess  this  training  program  (Fischer  &  Geiwitz,  1996).  The  results  were  promising. 
Experimental  participants  from  four  reserve  officer  training  corps  (ROTC)  courses  attained  terrain 
pattern  recognition  skills  comparable  to  a  sample  of  Army  Captains  who  had  completed  officer 
basic  and  advanced  training,  and  several  years  in  field  assignments. 

The  extended  description  of  the  tactical  pattern  recognition  training  of  Fischer  and  Geiwitz 
(1996)  reflects  a  recommendation  that  such  methods  should  be  applied  to  digital  technologies. 
Their  approach  to  pattern  meaning  centered  on  the  military  significance  of  each  of  40  terrain 
patterns,  including  their  tactical  advantages  and  disadvantages.  Their  approach  and  catalog  of 
patterns  for  Terrain  is  recommended,  but  additional  patterns  are  required  for  the  other  factors  of 
METT-T.  The  overall  METT-T  organization,  or  network,  of  the  patterns  to  be  trained  should 
reinforce  their  relevance  and  application  to  military  operations  and  related  training.  This  report’s 
focus  on  the  common  picture  objective  also  requires  that  pattern  recognition  training  methods 
address  patterns  as  depicted  on  a  C4I  display.  Benefits  derived  from  a  digital  application  of  pattern 
recognition  training  might  include  an  economy  in  training  delivery,  support,  assessment,  and 
management.  This  digital  conversion  would  also  support  training  method  recommendations  to 
supplement  pattern  recognition  skills,  such  as  automatic  processing  and  pattern  transformation 
training. 

Digital  technologies  may  be  the  only  realistic  medium  to  effectively  and  efficiently  employ 
the  training  methods  required  to  develop  automatic  processing  for  a  large  trainee  population. 
Methods  to  develop  automatic  processing  and  improve  performance,  for  example,  require  that 
trainees  experience  consistent  pairings  between  task  stimuli  and  responses,  and  relatively 
immediate  feedback  across  multiple  training  trials.  Inconsistent  pairings  and  inadequate  feedback 
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do  not  result  in  improved  performance  (Fisk  &  Eboch,  1989).  Notably,  automatic  processing 
methods  invariably  employ  a  visual  memory  search  paradigm  in  which  trainees  must  decide  if  an 
item  or  image  stored  in  memory  appears  in  a  displayed  set  of  items,  including  target  and  distractor 
items.  This  paradigm  meshes  well  with  a  focus  on  pictorial  communication  via  C4I  displays,  in 
general,  and  recognition  training,  in  particular. 

Given  their  documentation  (Fisk  &  Eboch,  1989),  the  authors  forgo  a  more  detailed 
discussion  of  training  methods  for  automatic  processing.  Their  methods  as  applied  to  map  reading, 
however,  are  particularly  applicable  to  METT-T  based  pattern  recognition  on  a  C4I  display.  An 
adaptation  of  these  methods  should  apply  training  guidelines  for  automatic  processing  to  avoid  the 
training  fallacies  documented  by  Schneider  (1986).  The  rationale  for  applying  automatic 
processing  methods  to  recognition  training  includes  the  more  general  benefits  cited  by  Rogers  et  al. 
(1997),  and  previously  noted.  Benefits  more  specific  to  pictorial  communication  objectives  might 
include  faster,  more  accurate,  and  less  variable  pattern  recognition  performance,  as  found  by  Fisk  & 
Eboch  (1989).  Their  work  also  demonstrated  that  trainees  with  automatic  processing  were  less 
susceptible  to  information  overload. 

One  might  hypothesize  similar  benefits  if  users  can  more  automatically  process  the  visual 
patterns  on  a  C4I  display.  Of  special  interest,  pattern  recognition  performance  might  be  less 
susceptible  to  degradation  typically  caused  by  larger  amounts  of  information  on  a  C4I  display.  In 
contrast  to  more  controlled  information  processing,  automatic  processing  is  a  relatively  capacity- 
free  process,  and  visual  search  for  key  patterns  on  a  display  becomes  markedly  non-serial.  With 
automatic  processing,  patterns  “pop  out”  of  the  display  (Fisk  and  Eboch,  1989,  p.  4). 

Pattern  Transformation.  Training  methods  should  also  apply  digital  technologies  to 
transform  and  augment  METT-T  patterns  depicted  by  C4I  systems  and  military  simulations.  Such 
capabilities  provide  unique  and  potentially  powerful  methods  to  help  trainees  shape  their  mental 
models  and  mental  processes  to  external  task  requirements.  The  methods  recommended  apply 
digital  transformation  capabilities  to  train  recognition  of  the  battlefield  patterns  on  C4I  displays. 

One  of  the  more  difficult  recognition  aspects  of  METT-T  battlefield  patterns  may  be  terrain 
appreciation.  It  is  particularly  difficult  to  perceive  the  elevation  of  terrain,  terrain  relief,  from  2-D 
map  representations  (Lickteig  &  Burnside,  1986).  Traditional  methods  for  training  terrain 
appreciation  and  pattern  recognition  rely  on  comparisons  between  2-D  and  3-D  perspective  views 
of  the  same  pattern  or  similar  patterns.  Presentation  formats  for  such  comparisons  include:  paper 
maps  and  illustrations,  as  used  by  Fischer  and  Geiwitz  (1996);  paper  maps  and  actual  terrain,  as 
used  in  field-settings;  and  paper  maps  and  terrain  photographs. 

A  more  innovative  method  for  training  terrain  appreciation  used  360-degree  photograph  sets 
from  various  field  locations  in  which  a  set  included  120  sequential  photographs  with  3-degrees  of 
separation  (Lickteig  &  Burnside,  1986).  The  photos  were  loaded  on  a  laser  disc  and  trainees 
scanned  a  location  by  using  a  joystick  that  controlled  scan  rate  and  direction  to  determine  their 
location  on  paper  maps.  This  same  report  also  describes  a  method  for  training  pattern  recognition 
that  used  cardboard  mockups  of  hills  painted  with  contour  elevation  lines.  Video  recordings  of  the 
mockups  were  made  from  perspective  and  overhead  views  and  then  loaded  onto  laser  disc. 
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Trainees  controlled  the  video  replays  using  joysticks  to  scan  between  2-D  and  3-D  perspectives  of 
the  mockups,  and  then  identified  similar  hilltop  patterns  on  paper  maps. 

However,  there  are  substantial  limitations  to  developing  and  delivering  both  the  traditional 
and  more  innovative  training  methods  just  described.  Field-based  training  methods  are  inefficient, 
requiring  physical  relocation  of  the  trainees  to  multiple  terrain  sites,  and  limited  to  the  types  of 
terrain  features  available  in  the  training  area.  More  traditional  map  and  photograph  comparisons 
are  often  based  on  a  small  number  of  photographs  that  provide  little  or  no  control  over  viewing 
perspective  or  visibility  conditions.  More  innovative  methods,  such  as  laser-disc  video,  overcome 
some  of  these  limitations,  but  entail  substantial  development  costs,  even  for  a  small  sample  of 
training  and  test  materials.  More  importantly,  the  training  effectiveness  of  most  map  reading  and 
terrain  appreciation  methods  is  limited  (Fischer  &  Geiwitz  1996;  Lickteig  &  Burnside,  1986). 

Digital  training  technologies  may  provide  more  efficient  and  effective  methods  for  training 
terrain  pattern  recognition  and  map  reading.  Efficiencies  are  expected,  in  part,  because  the  digital 
terrain  databases  in  more  advanced  C4I  systems  and  training  simulations  are  designed  to  store, 
display,  analyze,  and  transform  terrain  patterns.  Expansion  and  refinement  of  these  databases  and 
the  ability  to  process  and  manipulate  them  is  an  ongoing  effort.  Growing  reliance  on  simulation 
and  C4I  systems,  yoked  by  identical  digital  terrain  databases,  should  support  pattern  recognition 
training  in  simulated  battlefield  settings  anywhere  in  the  world  and  under  a  variety  of  realistic 
conditions.  More  effective  training  methods  are  the  primary  concern,  however. 

The  method  recommended  here  is  to  apply  the  ability  of  digital  technologies  to  pictorially 
transform  battlefield  patterns  to  train  pattern  and  situation  recognition  skills.  Battlefield  patterns 
such  as  terrain  relief,  mobility  corridors  and  intervisibility  are  readily  analyzed  and  portrayed  by 
digital  technologies.  Training  methods  designed  to  help  soldiers  appreciate  terrain  relief,  for 
example,  should  allow  trainees  to  visualize  and  control  transformations  between  2-D  and  3-D  views 
on  a  C4I  display.  These  transformations  might  resemble  a  progressive  “morphing”  of  images,  as 
commonly  seen  in  many  television  commercials,  between  2-D  and  3-D  perspectives  of  the  terrain. 
By  actually  displaying  the  otherwise  invisible  or  mental  process  of  transforming  a  flat  topographic 
map  into  visible  terrain  relief,  digital  technologies  can  pictorially  lead  the  trainee  through  the 
required  mental  transformations.  Allowing  trainees  to  control  the  pace  and  nature  of  such 
transformations  may  equate  to  cognitive  engagement  and  customization  of  training.  The  unique 
ability  of  digital  technologies  to  lead  the  process  of  transformation  may  approach  the  root  definition 
and  meaning  of  education  (educere— to  lead  forth). 

Situation  Recognition 

This  section  begins  by  examining  the  dynamic  nature  of  battlefield  situations  and  how 
situational  changes  are  reflected  in  METT-T  changes.  Methods  for  training  situation  recognition 
are  reviewed  and  particularly  how  these  methods  focus  on  the  trainee's  perceptual  processes.  Next, 
extensions  of  automatic  processing  theory  that  address  more  complex  tasks  such  as  situation 
recognition  are  reviewed,  and  recommendations  are  made  for  applying  these  methods  to  train  the 
recognition  of  battlefield  situations.  After  considering  how  digital  technologies  can  allow  trainees 
and  trainers  to  control  or  transform  battlefield  situations,  recommendations  are  provided  on  how 
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this  capability  might  be  used  to  train  the  recognition  of  battlefield  situations.  Again,  the  immediate 
concern  is  the  recognition  of  battlefield  situations  as  represented  on  C4I  displays,  but  any  acquired 
skills  should  generalize  to  other  representations,  such  as  analog  situation  maps,  and  actual 
battlefield  situations. 

Soldiers  must  be  able  to  recognize  and  respond  to  the  battlefield  situations  depicted  on  their 
C4I  displays.  For  related  METT-T  informational  elements,  multiple-feature  pattern  recognition 
might  be  regarded  as  a  relatively  focused  task,  or  part-task  requirement.  Across  METT-T  factors, 
situation  recognition  might  be  regarded  as  an  integrated  task,  or  whole-task  requirement.  The  total 
battlefield  situation  generally  exceeds  the  sum  of  its  METT-T  parts,  particularly  the  informational 
elements  displayed  by  the  C4I  system  at  any  isolated  moment  during  the  battle.  The  methods 
recommended  for  training  situation  recognition,  therefore,  stress  the  unique  ability  of  digital 
technologies  to  represent  and  accent  the  patterns  of  change  within  and  across  METT-T  factors  over 
time. 


Battlefield  situations  are  dynamic,  and  adequate  mental  and  micro  world  models  of  the 
situation  must  reflect  such  dynamics.  More  comprehensive  mental  models  go  beyond  a  depicted 
situation  or  state  to  include  the  dynamic  processes  underlying  changes  or  transitions  between 
situations  or  states.  A  mental  model  of  the  battlefield  situation,  therefore,  should  recognize  the 
underlying  state  changes  in  METT-T  factors  and  informational  elements  over  time  and  be  able  to 
project  future  changes.  A  recommended  approach  to  training  situation  recognition  stresses  that 
transitions  between  states  or  situations  may  be  the  most  important  aspect  in  the  development  of  a 
mental  model  (Kozma,  1991).  Recommended  training  methods  for  situation  recognition  allow 
users  to  visualize  and  manipulate  METT-T  parameters  to  explore  and  refine  their  internal  model  of 
a  battlefield  situation.  These  methods  are  directly  linked  to  the  battlefield  situation,  as  depicted  on 
a  C4I  display  or  digital  workstation. 

Methods  for  recognizing  battlefield  situations  emphasize  the  need  to  focus  a  trainee’s 
attention  on  those  factors  or  informational  elements  considered  typical  of,  or  critical  to, 
representative  situations.  For  example,  Klein  and  Calderwood  (1988)  state  such  training  should 
stress  sensitivity  to  the  critical  factors  that  distinguish  prototypical  situations,  in  order  to 
conceptualize  situations  quickly  and  accurately.  For  training  situation  recognition  for  firefighters, 
training  might  stress  identifying  the  type  of  fire.  For  maneuver  commanders,  training  might 
emphasize  enemy  composition  or  intent.  Other  types  of  critical  information  might  be  identified  by 
the  recommended  C4I  display  codes  that  targeted  “Critical  Information”  and  “Relevant 
Information”  based  on  FXXITP  exercises. 

Consistent  with  this  report's  focus  on  pictorial  communication,  Klein  and  Calderwood 
(1988)  stress  that  the  decision-making  of  experts  is  triggered  by  their  primed  recognition  of 
perceptual  patterns  in  complex  task  environments.  Similarly,  Federico’s  (1997)  work  on 
metacognitive  models  of  situation  assessment  suggests  that  perceptual  processes  are  the  primary 
determinant  of  tactical  decisions  and  actions. 

Accordingly,  the  methods  recommended  for  situation  recognition  focus  on  the  trainee’s 
perceptual  processes  and  the  ability  of  a  C4I  display  to  reflect  and  guide  those  processes.  To  aid 
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conceptualization  and  recognition  of  situations,  these  methods  direct  attention  to  METT-T  structure 
and  the  changes  in  METT-T  that  create  dynamic  battlefield  situations.  These  methods  should  also 
expand  and  integrate  lower-level  pattern  recognition  skills  in  relation  to  task  goals,  including 
situation  recognition.  They  should  also  foster  automatic  processing  skills  honed  by  consistent 
practice  via  digital  training  technologies. 

Automatic  Processing.  Extensions  of  automatic  processing  theory  and  findings  have 
demonstrated  its  applicability  for  training  complex  tasks  such  as  situation  recognition  (Fisk  & 
Eboch,  1989;  Fisk  &  Eggemeier,  1988).  The  key  method  to  such  extensions  is  the  identification  of 
higher-order  consistencies  and  their  structured  application  in  the  form  of  consistent  practice.  For  a 
more  detailed  description  of  the  interview  and  observation  methods  used  to  identify  consistent 
higher-order  components  see  Fisk  &  Eggemeier  (1988).  A  pertinent  application  of  such  methods  to 
identify  higher-order  consistencies  in  interface  display  and  controls  for  command  and  control  tasks 
was  demonstrated  by  Eggemeier,  Fisk,  Robbins,  &  Lawless  (1988).  Their  analysis  of  refueling 
scenarios  for  air  traffic  controllers  identified  consistent  task  elements  for:  (a)  estimating  direction, 
heading,  and  speed;  (b)  detecting  turn  initiation,  direction,  and  completion;  (c)  knowledge  of 
interface  controls;  and  (d)  knowledge  of  displayed  symbol  systems.  They  stress  such  consistencies 
are  inherent  to  real  world  activities  and  are  the  glue  that  binds  successful  systems. 

The  methods  of  Eggemeier  et  al.  (1988)  seem  directly  applicable  to  identifying  METT-T 
consistencies  for  training  situation  recognition  with  C4I  displays  and  related  digital  technologies.  In 
particular,  their  examples  illustrate  how  situation  recognition  may  depend  on  rapid  and  accurate 
processing  of  the  spatial  information  graphically  depicted  on  tactical  displays.  Such  consistencies 
may  provide  the  basis  of  expertise  in  perceiving,  comprehending,  and  projecting  current  and  future 
situations  (Endsley,  1995). 

To  identify  consistent  higher-order  components  related  to  situation  recognition,  a 
recommended  adaptation  of  the  Fisk  &  Eggemeier  (1988)  methods  would  use  interviews  and 
observations  directed  at  ground  forces  and  C4I  displays.  More  specifically,  expert  interviews  and 
observations  would  target  the  FXXITP  exercises  and  a  selected  C4I  system,  such  as  FBCB2.  These 
methods  should  result  in  a  set  of  higher-order  consistencies  for  command  and  control  tasks  that 
parallel  and  expand  the  types  identified  by  Eggemeier  et  al.  (1988)  for  air  traffic  controllers. 

Method  results  should  also  identify  higher-order  consistencies  in  the  interface  controls  and 
display  symbol  sets  of  the  subject  C4I  system  for  depicting  and  manipulating  METT-T  information. 
An  examination  of  required  enemy  information  during  FXXITP  exercises,  for  example,  might 
identify  consistent  task  elements  for  estimating  the  enemy’s  direction,  heading,  speed,  size,  activity, 
location,  unit  type,  and  equipment.  Similarly,  analysis  of  the  required  terrain  information  should 
help  identify  consistent  task  elements  for  recognizing  single-  and  multiple-feature  terrain  patterns 
such  as  cover  and  concealment,  mobility  corridors,  and  key  terrain. 

Once  task  consistencies  are  identified,  they  would  become  the  building  blocks  for  training 
situation  recognition  based  on  automatic  processing  methods  (Fisk  &  Eboch,  1989;  Fisk  & 
Eggemeier,  1988)  and  the  training  guidelines  outlined  by  Schneider  (1985).  Digital  training 
stations,  and  even  C4I  systems,  can  be  customized  to  consistently  map  their  representations  of  the 
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task  components  selected  for  training.  They  can  provide  repetitive  practice  that  automatically 
provides  training  analysis  and  feedback  based  on  the  user’s  response  to  those  training  stimuli. 
Grounding  these  methods  in  actual  C4I  systems,  FXX1TP  exercises,  and  METT-T  should  help 
develop  and  integrate  situation  recognition  skills  in  support  of  the  common  picture  objective. 

Situation  Transformation.  Training  methods  should  also  exploit  the  ability  of  digital 
technologies  to  transform  and  augment  the  situation  patterns  depicted  by  C4I  systems  and  digital 
workstations.  The  methods  recommended  here  stress  the  need  to  recognize  situational  transitions 
based  on  changes  in  METT-T  factors  over  time,  and  the  ability  to  project  future  changes.  These 
methods  should  help  trainees  build  more  accurate  and  temporally  comprehensive  mental  models  of 
a  battlefield  situation. 

Strategies  for  training  recognition  of  dynamic  situations  suggest  the  trainee’s  attention 
should  be  directed  to  variations  in  the  informational  elements  that  specifically  reflect  or  cause  a 
change  in  the  situation  (Kass,  Herschler,  &  Companion,  1991;  Klein  &  Calderwood,  1988).  One 
method  for  directing  attention  is  a  reduced  stimulus  environment  that  limits  the  informational 
elements  displayed  to  only  those  required  for  task  completion,  a  form  of  part-task  training.  In  a 
SIMNET  environment,  Kass  et  al.  found  that  pattern  recognition  training  (i.e.,  muzzle  flashes)  with 
a  minimum  of  extraneous  stimuli,  improved  the  ability  of  trainees  to  reliably  detect  this  critical 
change  later  in  a  more  complete  and  realistic  battlefield  environment.  This  method  illustrates  how 
digital  technologies,  such  as  virtual  simulation,  are  a  versatile  and  flexible  medium  for  directing 
attention  to  informational  features  that  reflect  and  cue  changes  in  a  battlefield  situation. 

A  more  compelling  example  of  part-task  training  methods  for  recognition  of  critical 
changes  is  the  work  of  Walker  and  Fisk  (1995)  for  training  football  quarterbacks  to  “read”  the 
defense  to  identify  open  receivers.  Their  training  stimuli  were  limited  to  eye-level  projections  of 
the  playing  field  during  the  few  seconds  a  quarterback  has  after  the  snap  to  identify  the  emerging 
patterns  of  coverage.  Player  movements  were  depicted  in  real-time  and  after  recognizing  the 
coverage  pattern,  the  quarterback  responded  by  identifying  the  designated  receiver.  These  methods 
could  be  readily  adapted  for  digital  simulation  and  C4I  systems.  For  example,  digital  method 
adaptations  for  recognizing  enemy  intent  in  dynamic  battlefield  situations  could  rapidly  depict  and 
reconfigure  enemy  activities  in  the  form  of  vignettes  extracted  from  FXXITP  exercises.  Trainees’ 
repeated  exposure  to  typical  and/or  critical  variations  in  enemy  activity  might  improve  their  ability 
to  read  enemy  intent. 

Related  training  methods  for  transforming  situational  changes  are  also  recommended.  As 
previously  noted,  animated  projections  of  a  friendly  unit’s  successive  locations  and  activities  might 
provide  a  useful  method  for  understanding  friendly  intent.  Similarly,  status  changes  in  key  items 
such  as  ammunition,  fuel,  equipment,  and  personnel  status  can  be  graphically  communicated  by  a 
C4I  display  (Leibrecht  et  al.,  1994). 

Perceptual  augmentation  is  another  method  for  directing  attention  to  changes  in  depicted 
situations.  Pertinent  examples  of  perceptual  augmentation  applied  to  a  C4I  display,  namely  the 
Extended  Joint  Surveillance  and  Attack  Radar  System,  are  provided  by  Kirlik,  Walker,  Fisk  and 
Nagel  (1996).  They  transformed  the  display  to  provide  additional  information  to  help  users 
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determine  when  displayed  enemy  vehicles  necessitated  action  by  the  trainee.  This  additional 
information  included  four  types  of  perceptual  augmentation:  (a)  locomotion  augmentation  that 
marked  which  areas  the  enemy  vehicle  could  and  could  not  enter,  (b)  weapons  range  augmentation 
that  encircled  the  vehicle’s  weapon  range,  (c)  penetrability  augmentation  that  differentially 
highlighted  friendly  vehicles  as  penetrable  or  not  penetrable  by  the  enemy  vehicle,  and  (d)  priority 
augmentation  that  superimposed  on  friendly  vehicle  icons  their  relative  importance.  Their  results, 
based  on  nonaugmented  trials,  showed  that  visual  augmentation  increases  the  acquisition  of 
decision-making  skills,  and  resists  performance  decrements  from  larger  threat  arrays. 

Extensions  of  such  perceptual  augmentation  methods  are  recommended  to  train  situation 
recognition  on  C4I  displays  and  digital  workstations.  Perceptual  augmentation  might  highlight 
significant  changes  in  the  informational  features  that  depict  the  battlefield  situation.  Currently, 
some  C4I  systems  highlight  the  icons  and  symbols  associated  with  recently  received  information, 
for  example.  For  training,  prerecorded  simulation  exercises  might  highlight  variations  in  key 
METT-T  features  or  patterns  during  preview  or  review  of  an  exercise.  Similarly,  a  form  of  process 
tracing  during  the  earlier  stages  of  training  might  model  and  highlight  the  role  and  actions  for  a 
trainee’s  duty  position  on  a  C4I  display  or  digital  workstation. 

Training  methods  that  compress  and  expand  time,  to  above-  and  below-real-time 
presentation  rates  are  also  recommended.  Digital  technologies  readily  manipulate  time  in  4-D 
microworld  models  and  simulations.  For  example,  Schneider’s  (1985)  methods  for  training  air 
traffic  controllers,  compressed  simulated  time  by  a  factor  of  1 00  to  train  visualization  of  flight 
patterns  (e.g.,  where  should  an  aircraft  turn).  Time  compression  methods  may  enhance,  or  make 
possible,  the  recognition  of  situational  changes  during  mission  segments  with  a  slower  pace.  Time 
compression  also  allows  trainees  to  experience  more  training  trials  than  possible  with  real-time 
representations  (Schneider,  1985).  Changes  on  a  battlefield,  however,  occur  at  very  uneven  rates. 
During  intense  segments  of  a  battle,  such  as  contact  and  engagement,  novice  trainees  might  benefit 
from  methods  that  slow  time  presentation  rates.  In  sum,  methods  that  allow  a  trainee  to  control 
time  and  space  patterns  in  microworld  representations  might  accelerate  the  transition  from  novice 
to  expert  skill  level  for  situation  recognition. 

Notably,  the  methods  documented  by  Walker  and  Fisk  (1995)  demonstrate  how  training 
can,  and  often  should,  combine  situation  transformation  methods,  such  as  stimulus  reduction,  with 
automatic  processing  methods.  The  ability  of  digital  technologies  to  automate  many  of  the  more 
exacting  training  development  steps  required  for  automatic  processing  and  situation  transformation 
methods  supports  a  more  extensive  integration  of  these  methods  to  train  the  recognition  of  tactical 
patterns  and  battlefield  situations. 


Maintain  the  Common  Picture 

This  section  identifies  several  key  issues  related  to  maintaining  the  common  picture  in 
particular,  and  training  for  pictorial  communication  and  information  management  in  general.  Basic 
research  issues  for  information  management  training  include  how  to  provide  the  training,  assess 
performance,  and  give  feedback  in  an  effective  and  efficient  manner.  For  this  report,  these  issues 
are  summarized  as  the  need  to  develop  a  digital  communications  model  of  the  required  information 
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exchanges  within  a  unit,  such  as  a  battalion  or  brigade.  A  second  issue  raised  is  the  need  to 
integrate  this  communications  model  with  related  digital  technologies,  namely  warfighting 
simulations  and  C4I  systems,  to  provide  training  in  realistic  job  and  battlefield  settings.  The  third 
and  final  issue  considered  is  to  determine  how  to  apply  this  integrated  technology  to  address  multi¬ 
level  training. 

Corresponding  method  recommendations  for  each  of  these  issues  are  provided.  These 
method  recommendations  are  based  on  reviewed  methods  for  training  information  management 
skills  and  applying  digital  models  to  make  that  training  more  effective  and  efficient.  Before 
providing  these  method  recommendations,  therefore,  background  training  issues  and  methods 
related  to  information  management  are  reviewed. 

The  battlefield  situation  depicted  on  a  C4I  display  is  a  collaborative  soldier-computer 
product.  Maintaining  the  common  picture,  a  current  and  accurate  depiction  of  a  dynamic 
battlefield,  is  a  collaborative  soldier-computer  process.  This  process  entails  a  host  of  intersystem 
and  intrasystem  information  exchanges,  and  is  a  challenging  information  management  task.  With 
the  less  advanced  C4I  systems  currently  fielded,  the  challenge  falls  heavily  on  soldiers.  Training 
methods  are  sorely  needed  to  help  soldiers  maintain  a  common  picture  of  the  battlefield  on  their  C4I 
displays. 


“What  is  the  process  of  maintaining  the  common  picture?”  was  a  key  question  raised  by  the 
Army’s  Joint  Venture  Campaign  Plan  to  leverage  digital  technologies  for  force  and  training 
development  (DA,  1996a).  Recall  that  one  answer  from  Focused  Dispatch  was  swivel-chair 
integration— manual  transfers  from  one  C4I  system  to  another— due  to  incompatible  C4I  systems 
(U.S.  Army  Armor  Center,  1996).  A  more  useful  answer  may  require  a  detailed  analysis  and 
understanding  of  the  communications  required  between  combatants,  supporters,  and  their  C4I 
systems. 

The  Army’s  experience  with  digital  technologies  has  strongly  reinforced  the  need  to  better 
manage  information.  The  AWEs  have  repeatedly  found  that  digital  C4I  systems  can  impose 
substantial  and  detrimental  amounts  of  information  on  warfighters  and  supporters  (U.S.  Army 
Armor  Center,  1994;  1996).  Similarly,  early  testing  on  the  Longbow  Apache  helicopter  that  is 
leading  Army  aviation  into  the  information  age,  concluded  that  increased  information  from  digital 
sensors,  displays,  and  communications  makes  information  management  a  critical  combat  skill 
(Snook,  1997). 

Across  private,  commercial,  and  military  sectors,  digital  technologies  are  becoming 
synonymous  with  information  overload.  In  essence,  digital  technologies  expand  our  access  to,  and 
accountability  for,  more  information.  Given  our  inexorable  reliance  on  and  alliance  with  digital 
information  technologies  (Negroponte,  1995),  we  must  devise  better  ways  to  manage  information 
and  train  information  management.  “Better  education  and  training,  devoted  to  information 
processing  under  stress  and  in  environments  characterized  by  uncertainty,  are  needed  to  develop  the 
necessary  skills  to  handle  these  information-rich  situations”  (Alberts,  1996,  p.  32). 
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The  military  has  established  methods  and  techniques  for  managing  information  and 
workload,  such  as  standing  operating  procedures  (SOPs),  drills,  planning,  rehearsal,  and  training 
(National  Research  Council,  1997).  These  standard  information  management  techniques  should  be 
adopted  and  adapted,  where  appropriate,  as  additional  methods  are  developed  to  manage  digital 
information.  For  example,  Focused  Dispatch  developed  a  prototype  digital  SOP  that  documents 
numerous  procedures  for  managing  digital  information  (U.S.  Army  Armor  Center,  1996).  The 
primary  concern  here,  however,  is  devising  new  digital  methods  and  techniques  for  training 
information  management  and  maintaining  a  common  battlefield  picture. 

One  example  of  a  part-task  method  for  training  digital  information  management  is  the  use 
of  automated  message  servers  to  send  digital  reports  to  C4I  system  operators  (Lickteig  &  Emery, 
1994).  Methods  for  this  example  included  SME  development  of  message  sets  that  varied 
information  amount  and  relevance  for  an  approved  training  scenario  to  provide  operational  realism. 
During  an  exercise,  these  messages  were  automatically  delivered  at  a  realistic  rate  to  each 
participant’s  C4I  system  located  in  a  virtual  tank  simulator.  Information  management  tasks  required 
the  participants  to  process  the  messages  for  a  subsequent  situational  awareness  task,  and  also  relay 
as  appropriate  any  of  the  messages  received  to  surrogate  members  of  their  unit  at  higher,  lower,  and 
adjacent  echelons.  A  follow-on  effort  developed  and  implemented  a  prototype  information 
management  training  program  for  this  method,  and  automated  a  performance  feedback  approach 
based  on  expert  analysis  (Winsch  et  al.,  1994).  More  recent  adaptations  have  extended  this 
individual  training  method  to  small  group  staff  training  (Koger  et  al.,  1998). 

Another  example  is  the  team  model  trainer  (TMT)  developed  by  Duncan  et  al.  (1996). 
Although  the  TMT  addressed  predominantly  verbal  communications,  its  innovative  methods 
exemplify  how  digital  technologies  might  be  applied  to  training.  Based  on  an  expert  model  of  team 
member  communications  and  activities  in  a  Combat  Information  Center,  TMT  was  used  to  train 
team  members  individually,  prior  to  small  group  and  collective  exercises. 

The  TMT  runs  on  a  stand-alone  digital  workstation.  In  addition  to  simulating  team 
communications  and  actions,  a  TMT  workstation  provides  additional  information  about  the  roles  of 
team  members,  team  relationships,  the  information  requirements  of  team  members,  and  the 
temporal  patterns  of  team  communication.  One  product  from  the  TMT  model  is  a  team 
communication  matrix  that  identifies  communication  channels  and  content  among  team  members. 
A  related  product,  of  special  interest,  is  a  simulated  counterpart  of  this  matrix  that  visually  depicts 
the  flow  and  pace  of  communications  among  team  members  on  the  trainee’s  TMT  workstation. 

The  TMT  can  run  as  either  a  closed-  or  open-loop  model  that  supports  observer  or 
performer  mode  training,  respectively.  As  an  observer,  the  trainee  receives  a  visual  and  aural 
demonstration  of  expert  team  performance  on  a  training  device  that  emulates  an  operator’s  display 
and  plays  prerecorded  verbal  communications.  As  part  of  this  observer  training,  TMT  mimics  the 
actions  and  communications  for  the  trainee’s  duty  position.  As  a  performer,  the  trainee  can  assume 
any  team  member  role,  and  then  respond  with  the  actions  and  communications  for  that  duty 
position  as  the  model  simulates  the  actions  and  communications  of  other  team  members.  The 
trainee  can  also  start  and  stop  the  simulation  at  any  time  for  help  or  additional  information,  or 
replay  the  scenario  to  review  or  rehearse  selected  events. 
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After  selecting  a  team  position  and  mode  (observer  or  performer),  the  trainee  receives  and 
reviews  a  situation  update  tailored  to  the  position  selected.  The  trainee  then  selects  an  event  from  a 
scenario  event  list  provided  on  the  display  and  the  simulation  unfolds  from  that  point  of  the 
scenario.  During  the  simulated  scenario,  the  trainee’s  TMT  display  depicts  and  identifies  the 
stations  for  all  six  team  members  in  the  Combat  Information  Center.  Team  actions  and 
communications  are  depicted  as  an  interactive  graphic  in  which  each  duty  position  depicted  on  the 
trainee’s  workstation  highlights  as  that  position  communicates  or  acts.  The  display  also  depicts  a 
timeline  of  the  scenario  with  appropriately  sequenced  action  and  communication  boxes  that 
highlight  at  the  appropriate  time. 

A  final  example  of  methods  pertinent  to  training  and  maintaining  a  common  picture  is  an 
ongoing  effort  to  improve  after  action  reviews  (AARs)  for  units  equipped  with  C4I  systems  (Brown, 
et  al.,  1996).  This  work  is  designed  to  automate,  at  least  partially,  training  analysis  and  feedback 
methods  for  the  collective  AARs  that  generally  follow  unit  level  training  exercises.  Two  pertinent 
method  aspects  of  this  work  are  noted. 

First,  their  methods  capitalize  on  the  composite  database  that  underlies  a  C4!  display.  To 
avoid  the  labor  and  delays  associated  with  videotape  comparisons  of  the  C4I  displays  for  AARs, 
their  design  concept  stresses  automated  reconstructions  and  comparisons  of  C4I  display  content. 
Second,  these  comparisons  target  the  graphical  features  depicted  on  C4I  displays.  This  work  is  a 
basis  for  this  report's  working  example  of  automated  pictorial  comparisons  from  instrumented  C4I 
systems.  Method  recommendations  on  multi-level  training  will  examine  how  automated  pictorial 
comparisons  might  be  used  for  individual,  small  group,  and  collective  training. 

Develop  a  Communications  Model 

Fundamental  research  issues  for  information  management  training  include  how  to  provide 
the  training,  assess  performance,  and  give  feedback  in  an  effective  and  efficient  manner.  Providing 
training  is  a  multi-level  requirement  that  includes  individual,  small  group  and  collective  training 
requirements.  Performance  standards  of  required  information  exchanges  are  needed  to  assess 
performance.  Given  standards  in  adequate  detail  and  format,  the  question  becomes  how  to  apply 
such  standards  more  easily  and  evenly  to  assess  training  and  provide  feedback? 

A  recommended  method  to  address  these  issues  is  tailored  to  this  report's  focus  on  training 
pictorial  communications:  develop  a  common  picture  communications  model.  This 
recommendation  equates  to  a  computer-based  model  that  simulates  the  required  information 
exchanges  among  all  unit  members  during  the  course  of  a  training  exercise.  This  model  does  not 
yet  exist,  but  its  primary  features  are  described  to  clarify  its  training  applications.  To  aid  this 
description,  the  term  Common  Picture  Communications  Model  (CPCM)  is  coined  to  identify  this 
model.  Although  specification  of  CPCM  is  not  attempted  here,  its  design  is  based  heavily  on  the 
communication  models  of  Duncan  et  al.  (1996)  and  Laughery  and  Corker  (1997). 

As  a  basis  for  CPCM  design  and  development,  structured  and  realistic  tactical  scenarios 
are  recommended,  such  as  the  FXXITP  exercises.  The  detailed  specification  of  the  FXXITP 
exercises  and  their  delineation  of  scripted  events  during  the  exercise  provide  a  solid  foundation  for 
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building  a  communications  model.  The  model  should  initially  be  a  battalion-level  development, 
with  subsequent  extension  to  brigade  level.  A  FXXITP-based  approach  also  capitalizes  on  an 
earlier  recommendation  to  use  these  exercises  for  eliciting  user  information  requirements.  Recall 
that  these  knowledge  elicitation  methods  were  designed  to  identify  required  information  exchanges 
within  a  unit  and  included  both  C4I  digital  and  voice  radio  communications.  Although  digital 
exchanges  are  key  to  CPCM,  it  should  include  voice  radio  communications  for  information 
requirements  not  directly  supported  by  the  C4I  systems  modeled,  for  example  FBCB2. 

In  essence,  the  required  information  exchanges  identified  by  SMEs  would  be  implemented 
in  a  computer-based  model  that  simulates  the  flow  of  information  exchanges  among  the  FXXITP- 
based  duty  positions  equipped  with  C4I  systems.  Based  on  SME  inputs,  the  CPCM  represents  an 
expert  model  that  represents  an  optimal  flow  of  information  among  the  members  of  the  unit.  This 
multipurpose  model  could  be  used  to  demonstrate  information  exchanges  and  to  generate  criterion- 
based  product  and  process  examples  for  training  analysis  and  feedback  on  C4I  battlefield  depictions. 

Although  an  expert  version  of  the  CPCM  has  unique  advantages,  it  entails  substantial  costs. 
Therefore,  a  more  automated  method  for  developing  CPCM  is  briefly  described.  Virtual  simulation 
has  developed  and  effectively  used  software  that  emulates  the  information  exchanges  between 
modular  semi-automated  force  (ModSAF)  entities,  such  as  combat  and  support  vehicles,  and 
delivers  these  intersystem  communications  to  soldiers  equipped  with  C4I  systems  (Elliott,  Sterling 
&  Lickteig,  1998;  Leibrecht  et  al.,  1994).  In  theory,  simulation  files  designed  in  accordance  with 
FXXITP  structured  exercises  could  be  run  with  completely  unmanned  simulated  entities  to  develop 
a  “machine”  version  of  the  CPCM.  Realistically,  soldier  and  SME  inputs  would  still  be  required  to 
complete  and  validate  this  version  of  a  communications  model.  However,  the  costs  for  developing 
a  machine  version  of  this  model  should  be  substantially  lower  than  for  an  expert  version. 

Additional  tradeoffs  between  different  model  versions,  including  constructive  simulation 
generation,  should  be  considered,  but  are  not  in  this  report.  Subsequent  discussion  of 
recommended  CPCM  training  applications  is  independent  of  version,  unless  otherwise  noted. 

The  CPCM  should  run  on  stand-alone  and  networked  digital  workstations.  These  stations 
should  emulate  both  the  interface  and  controls  for  a  designated  C4I  system,  and  for  related 
computer-based  training.  For  stand-alone  configuration,  the  TMT  design  provides  good  examples 
of  how  a  CPCM  workstation  might  be  applied  to  training  (Duncan  et  al.,  1996).  The  CPCM  station 
would  support  both  observer  and  performer  trainee  modes  for  any  duty  position  included  in  the 
unit-based  FXXITP  training  exercise. 

Integrate  Digital  Technologies 

Another  key  method  for  applying  digital  technologies  to  train  and  maintain  the  common 
picture  is  to  link  digital  training  workstations  and  C4I  systems  with  the  warfighting  simulations  the 
Army  currently  uses  for  training.  Constructive  and  virtual  simulation  are  inherently  digital,  and 
live  simulation  with  C4I  equipped  units  is  becoming  digital.  Digital  technologies  can  effectively 
integrate  these  simulation  domains  (Cosby,  1995).  Digital  technologies  can  also  effectively  link 
Army  simulations  with  other  digital  training  applications,  and  the  methods  recommended  stress 
simulation  links  to  the  CPCM.  Integration  of  the  CPCM  to  Army  training  simulation  should 
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substantially  broaden  its  training  potential  and  provide  a  foundation  for  developing  a  digital 
training  environment. 

Constructive  Simulation 


Relative  to  other  Army  training  simulations,  constructive  simulation  is  more  a  closed-loop 
model  for  training  applications.  Training  benefits  from  closed-loop  models  include  their  ability  to: 
(a)  stimulate  the  training  response  by  serving  as  an  exercise  driver;  (b)  pace  training,  including 
above-  and  below-real  time;  (c)  represent  more  realistic  job  settings;  and  (d)  model  performance. 
For  observer  mode  training,  a  direct  time-  and  event-link  between  CPCM  and  a  constructive 
simulation  could  provide  the  trainee  a  broader  and  more  meaningful  microworld  context  of  battle 
simulation.  For  CPCM  performer  mode  training,  a  constructive  depiction  of  related  battlefield 
activities  could  stimulate  the  responses  required  by  the  trainee  to  maintain  a  common  picture  of  the 
simulated  battlefield  on  own  and  others  C4I  displays. 

Virtual  Simulation 


The  soldier-in-the-loop  nature  of  virtual  simulation  is  a  potent  environment  for  training  and 
assessing  soldier  performance  (Fletcher,  1994).  Human  interaction  can  dramatically  illustrate  and 
reinforce  the  value  of  coordination  and  communication,  the  value  of  maintaining  a  common  picture 
of  the  battlefield.  More  powerful  training  applications  are  possible  with  a  CPCM  model  that  runs 
on  virtual  simulation  networks,  such  as  SIMNET,  and  is  synchronized  with  the  entities  simulated 
on  virtual  battlefields.  Notably,  compatibility  with  virtual  networks  is  almost  a  “given”  for 
machine  versions  of  CPCM  generated  in  virtual  simulation.  The  CPCM  stations  could  also  be 
hosted  on  simulated  C4I  systems  in  training  simulators  to  immerse  trainees  in  more  realistic 
battlefield  settings  (Alluisi,  1991). 

Individual  training  might  use  the  CPCM  to  send  reports  from  a  FXXITP  exercise  to  the 
trainee  in  a  C4I  simulator  who  would  process  the  information  received  and  simultaneously  perform 
other  operational  tasks  during  the  simulated  exercise.  Small  group  training  might  network  C4I 
systems  in  multiple  simulators  to  the  CPCM.  Training  exercises  could  require  small  groups  of 
trainees  to  interactively  maintain  relevant  portions  of  the  common  picture  on  own  and  others  C4I 
displays.  Exercise  extensions  might  again  require  trainees  to  simultaneously  perform  their  other 
operational  tasks  during  the  simulated  exercise.  Collective  training  applications  of  the  CPCM 
might  address  before  and  after  phases  of  a  collective  training  exercise.  Before  a  training  exercise,  a 
unit  might  use  the  CPCM  to  rehearse  the  flow  of  critical  or  priority  information  by  element,  source, 
and  path.  After  the  exercise,  the  CPCM’s  examples  of  information  flow  and  C4I  common  picture 
snapshots  could  support  automated  training  analysis  and  feedback  for  collective  AARs. 

Live  Simulation 


Live  simulation  may  be  the  Army’s  most  demanding  and  realistic  training  venue  and  the 
most  difficult  for  training  delivery,  analysis,  and  feedback.  Moreover,  a  recent  analysis  of  live 
force  training  requirements  on  the  digital  battlefield  notes  that  C4I-based  training  increases  the 
training  analyst’s  load  and  forces  a  faster  feedback  process  to  provide  meaningful  training 
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experience  (U.S.  Army  Training  Support  Center,  1996).  In  particular,  this  analysis  stresses  that 
Xxairimg  feedback  overload  must  be  avoided  for  C4I-based  live  training.  The  computer-based  nature 
of  C4I  systems  could  embed  training  into  digital  combat  and  support  systems  to  enable  on-board 
training  for  live  simulation  exercises  (Morrison  &  Orlansky,  1997).  The  CPCM  might  represent  a 
module  in  an  embedded  training  package  for  C4I  systems.  The  need  for  C4I  systems  to  include  a 
simulated  model  for  training  and  mission  rehearsal  is  frequently  cited  (Coe,  Madden,  Mengel,  & 
Wright,  1997).  As  both  a  stand-alone  and  networked  module,  this  embedded  information 
management  model  could  be  used  to  train  pictorial  communication. 

Synthesized  simulations— a  mix  of  constructive,  virtual,  and  live  simulation-might  use 
CPCM  linkages  to  help  integrate  training  delivery,  analysis,  and  feedback  across  diverse  units  in  a 
variety  of  ways.  First,  the  overall  METT-T  organization  of  the  methods  provides  a  common  and 
coherent  framework  to  support  training  before  and  after  simulated  exercises.  Second,  feedback 
overload  might  be  avoided  by  presentation  formats  based  on  pictorial  comparisons  of  actual  versus 
model  performance.  Also,  the  common  picture’s  integration  requirements  might  effectively 
illustrate  and  reinforce  the  informational  dependencies  between  synthetically  linked  trainees  who 
might  be  physically  dispersed  at  remote  training  sites. 

Develop  Multi-Level  Training 

Information  management  training  addresses  relatively  high-level  skills,  particularly  with  the 
increased  amount  and  complexity  of  information  introduced  by  digital  technologies,  such  as  C4I 
systems.  It  presumes  lower-level  C4I  skills  such  as  basic  procedural  skills  to  operate  or  manipulate 
the  system  via  the  interface  controls.  It  presumes  the  recognition  skills  to  understand  the  battlefield 
patterns  and  situations  depicted  on  a  C4I  display.  Information  management  skills  should  also 
extend  to  understanding  the  possibilities  of  a  C4I  system  or  systems  for  mission  accomplishment 
(Gattiker,  1990).  While  the  CPCM  might  be  used  to  train  all  of  these  supporting  skills,  the  primary 
focus  here  is  on  training  to  maintain  a  common  picture. 

By  design,  the  CPCM  model  provides  training  analysis  and  feedback  on  trainees’  ability  to 
maintain  a  common  picture.  As  a  model  of  unit-level  required  C4I  information  exchanges,  the 
CPCM  database  should  closely  approximate  the  composite  database  of  soldier-computer 
interactions  required  to  maintain  an  accurate  depiction  of  the  battlefield  situation  during  the  training 
exercise.  The  methods  recommended  stress,  therefore,  that  discrepancies  between  the  trainees’  and 
the  model’s  process  and  products  could  be  automatically  detected  and  graphically  highlighted.  The 
following  method  recommendations  integrate  and  apply  the  digital  technologies  of  CPCM  and 
Army  warfighting  simulations  to  individual,  small  group,  and  collective  training. 

Individual  Training 

The  potential  of  applying  digital  technologies  such  as  CPCM  to  individual  training  on 
digital  systems  seems  high  compared  to  TMT’s  application  to  analog  training.  For  training  to 
maintain  a  common  picture  on  a  C4I  display,  the  required  information  exchanges  are  almost 
exclusively  digital.  The  multipurpose  nature  of  a  digital  workstation  for  such  training  is 
exemplified  by  its  ability  to  configure  as  a  generic  training  workstation  and/or  a  C4I  interface. 
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More  specifically,  a  portion  of  the  CPCM  workstation  could  be  configured  to  mirror  the  display 
and  controls  of  the  trainee’s  C4I  system,  such  as  the  FBCB2  system.  This  area  of  the  CPCM 
workstation  would  emulate  the  functionality  of  the  trainee's  C4I  interface,  for  both  observer  and 
performer  modes. 

For  observer  mode  training,  the  CPCM  might  initially  provide  trainees  an  overview  that 
epitomizes  the  information  management  requirements  and  activities  for  their  duty  position  and 
other  members  of  their  unit.  This  training  would  require  configuration  of  the  CPCM  display  to 
depict  multiple  duty  stations  or  platforms  sending  and  receiving  messages.  Vehicle  and  unit  icons 
might  be  tactically  arrayed  on  a  C4I  map  display  and  move  in  accordance  with  the  exercise's 
dynamic  battlefield  situation.  The  CPCM  interface  could  visually  depict  and  highlight  the  flow, 
pace,  and  content  of  communications  among  the  vehicles  and  duty  positions  depicted  during  a 
FXXITP  exercise. 

Configured  to  emulate  a  C4!  interface,  on  the  other  hand,  a  CPCM  workstation  could  mimic 
the  human-computer  interactions  required  for  the  trainee’s  selected  duty  position.  As  CPCM 
messages  from  a  simulated  FXXITP  exercise  were  received,  for  this  observer  mode  training,  the 
simulated  interface  would  open  the  messages  and  post  associated  graphic  content  to  the  tactical 
display’s  battlefield  representation.  When  appropriate,  the  simulated  interface  might  also 
manipulate  map  scale  and  zoom  levels  to  provide  an  adequate  situational  context  for  message 
processing.  As  C4I  messages  were  sent  by  the  CPCM  from  the  trainee's  workstation,  the  interface 
would  continue  to  mimic  the  soldier-computer  interactions  required  for  distributing  this 
information.  If  this  CPCM  work  station  was  embedded  in  a  virtual  simulator  and  linked  to  a  virtual 
exercise,  then  the  required  information  exchanges  could  be  demonstrated  in  a  task-based  battlefield 
setting  that  the  trainee  could  stop,  review,  and  replay  as  desired. 

For  performer  mode  training,  the  simulated  C4I  interface  would  still  receive  and  depict 
CPCM  communications  but  require  trainees  to  make  the  inputs  for  their  respective  duty  position. 
The  trainee  would  make  all  the  human  interactions  required  to  receive  and  display  battlefield 
information  on  the  simulated  C4I  display.  At  the  same  time,  the  trainee  would  perform  the 
procedures  required  to  originate  or  forward  communications,  as  appropriate,  to  other  surrogate 
members  of  the  unit. 

The  CPCM  workstation  design  should  also  customize  training  to  address  particular 
information  management  needs.  The  trainee’s  designated  duty  position  and  the  information 
exchanges  relevant  to  that  position  might  be  a  primary  training  focus.  Information  exchanges 
related  to  other  key  duty  positions,  such  as  the  CCIR  or  PIR,  might  be  a  more  specific  focus  of 
training.  Customization  could  also  allow  a  METT-T  breakout  of  information  types,  for  example,  to 
selectively  highlight  the  flow  of  enemy  information  between  the  trainee’s  duty  position  and  key 
superior  and  subordinate  positions  during  the  course  of  a  FXXITP  exercise.  Similarly,  an  event- 
based  focus  on  required  communications  might  help  the  trainee  anticipate  and  understand  workload 
variations  during  the  course  of  a  FXXITP  exercise. 

Automated  training  analysis  routines  could  compare  the  trainee’s  performance  with  that  of 
the  CPCM,  and  prepare  performance  feedback  products  available  during  and  after  the  training 
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exercise.  Training  objectives  and  feedback  should  stress  how  unit  members  are  dependent  upon 
one  another  to  maintain  an  accurate  and  relevant  picture  of  the  battlefield  situation.  Feedback 
formats  should  pictorially  highlight  the  unit’s  dependence  on  the  information  controlled  by  the 
trainee.  Feedback  might  provide  pictorial  comparisons  between  the  trainee's  C4I  display  and  the 
model-generated  displays  of  other  surrogate  unit  members.  Other  feedback  examples  might 
highlight  information  that  should  have  been  available  to  the  trainee’s  superiors  and  subordinates 
that  was  not  provided  by  the  trainee,  or  compare  trainee’s  versus  model  timelines  for  exchanging 
information. 

Small  Group  Training 

Collaborative  requirements  to  maintain  a  common  picture  of  the  battlefield  situation  should 
be  stressed  as  training  progresses  from  individual  to  small  1  group  level.  Unlike  TMT,  networked 
CPCM  workstations  should  link  multiple  trainees  in  small  group  training  exercises.  Many  of  the 
methods  recommended  for  individual  training  can  be  readily  extended  to  include  multiple  trainees. 
For  small  group  training  the  CPCM  could  depict  a  subset  of  other  duty  positions  to  model 
information  exchanges  within  a  designated  group.  Training  methods  could  apply  the  CPCM  open- 
model  to  allow  two  or  three  trainees  to  co-participate  in  the  same  information  management 
exercises.  As  an  automated  message  server,  the  CPCM  would  send  digital  reports  to  this  small 
group  of  trainees  who  would  be  required  to  originate  and  relay,  as  appropriate,  C4I-based 
communications  to  each  other  as  well  as  other  surrogate  members  of  their  unit. 

Training  objectives  for  small  groups  should  stress  information  exchanges  required  between 
the  current  trainees.  Accordingly,  training  analysis  and  feedback  might  concentrate  on  unmet 
information  requirements  or  delayed  communications  based  on  the  information  management 
performance  of  the  trainees.  Feedback  formats  might  include  display  snapshot  or  timeline 
comparisons,  or  instances  where  trainees  relayed  redundant  information  to  other  members  of  their 
unit.  As  with  individual  training,  all  feedback  might  be  structured  by  phases  and  events  from  the 
FXXITP  exercise  used  for  training. 

Collective  Training 


Collective  training  generally  extends  the  training  audience  to  include  all,  or  a  substantial 
portion  of,  unit  members  interacting  in  a  simulated  operational  setting.  Unit  size  and  exercise 
objectives  may  vary  substantially,  however,  for  a  collective  training  audience.  The  VTP-based 
exercises,  for  example,  include  platoon  and  company  tables  that  are  mission  segments  extracted 
from  larger  unit  lull-mission  operations.  Similar  tables  are  recommended  to  focus  training 
objectives  and  feedback  on  the  information  management  requirements  for  units  equipped  with  C4I 
displays.  The  expert  analysis  of  FXXITP-based  information  requirements,  previously  discussed, 
should  help  formulate  training  objectives  and  structure  collective  training  exercises  and  exercise 
segments  directed  at  maintaining  a  picture  of  the  battlefield  situation  on  trainees’  C4I  displays. 

Collective  training  provides  an  ideal  forum  to  understand  and  practice  the  collaborative 
process  of  managing  information.  To  minimize  disruption  of  a  collective  exercise,  however, 
training  methods  for  information  management  that  apply  before  and  after  the  unit  conducts  a 


53 


mission  or  table  are  recommended.  Prior  to  a  collective  exercise,  the  CPCM  could  support  training 
objectives  for  information  management  products  and  procedures.  Large-scale  digital  displays  with 
CPCM  could  demonstrate  and  illustrate  collective  process  and  product  examples  for  managing 
information  to  maintain  a  picture  of  the  battlefield.  Methods  for  demonstrating  process  and 
illustrating  products  could  be  used  to  refresh  or  rehearse  information  management  skills,  prior  to 
the  collective  exercise. 

For  example,  a  key  training  requirement  for  information  management  is  an  appreciation  of 
the  overall  digital  communications  network  structure.  Past  digital  training  experiences  have 
stressed  that  this  is  a  critical  but  neglected  aspect  of  training,  and  that  alphanumeric  routing 
matrices  are  not  an  effective  format  for  conveying  such  training  (Elliott,  Sanders,  &  Quinkert, 

1996).  Collective  training,  in  particular,  might  use  CPCM  to  pictorially  illustrate  the  flow  of 
information  across  the  unit’s  digital  architecture  and  underscore  its  importance  for  successfully 
completing  the  upcoming  training  exercise.  Key  decision  and  coordination  points  in  the  training 
exercise,  for  example,  might  be  identified  and  depicted  in  their  anticipated  battlefield  context  on  a 
large-scale  Cl  display.  The  information  requirements  associated  with  each  of  these  points  by 
element,  source,  and  path  might  be  identified  and  pictorially  highlighted. 

After  the  exercise,  digital  training  applications  might  adapt  some  of  the  automated  training 
analysis  and  feedback  methods,  previously  described,  for  unit-level  AARs.  Methods  for  AARs 
should  stress  automated  reconstructions  and  comparisons  of  C4I  displays  within  the  unit,  or  with 
displays  generated  by  the  CPCM  model.  These  methods  should  directly  target  the  graphical 
features  depicted  on  Cl  displays. 

Product  feedback  examples  include  automated  snapshot  comparisons  between  the  Cl 
displays  of  any  unit  trainees  at  any  time  during  the  training  exercise  that  might  disclose  important 
discrepancies,  such  as  the  location  of  friendly  and  enemy  vehicles,  obstacles,  or  contaminated  areas. 
Product  examples  might  also  include  automated  comparisons  of  Cl  display  content  based  on  unit 
performance  with  CPCM  products.  Overall,  product  feedback  should  stress  pictorial  formats  that 
automatically  highlight  potential  problems  in  the  unit’s  effort  to  maintain  a  picture  of  the 
battlefield. 

Process  feedback  might  address  problems  related  to  discrete  events  at  key  moments  during 
the  exercise  or  to  the  more  continuous  flow  of  information  across  the  exercise.  The  information 
flow  formats  used  before  the  exercise  to  model  information  exchanges  could  be  compared  with  the 
unit’s  actual  performance  during  the  exercise.  Or  for  information  elements  identified  as  critical  by 
the  unit  before  the  training  exercise,  the  AAR  could  automatically  illustrate  the  flow  of  this 
information  during  the  exercise.  Information  exchanges  supporting  key  decision  points,  for 
example,  during  the  exercise  could  be  graphically  illustrated  on  a  dynamic  timeline  by  information 
element,  source,  and  path.  Overall,  process  feedback  should  stress  pictorial  formats  that  depict  the 
process,  the  problems  identified,  and  sample  solutions. 
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Method  Summary 


This  method  section  identified  three  main  areas  of  research  directed  at  the  pictorial 
communication  of  battlefield  situations  on  C4I  displays.  These  research  areas  were  centered  on 
common  picture  training  and  evaluation  requirements:  define,  communicate,  and  maintain  a 
common  picture  of  the  battlefield.  For  each  of  these  areas,  a  set  of  research  issues  related  to  the 
requirement  were  identified,  and  then  corresponding  training  and  evaluation  methods  were 
recommended  to  help  address  the  issues  raised,  see  Table  1 . 

The  methods  recommended  were  based  on  the  authors'  review  of  documented  methods  and 
the  relatively  unique  capability  of  digital  information  systems,  such  as  instrumented  C4I  systems 
and  military  warfighting  simulations,  to  synthesize  training  and  evaluation.  Background 
information  provided  more  detailed  description  of  and  reference  to  the  methods  recommended. 

The  method  recommendations  adapted  a  relatively  integrated  set  of  training  and  evaluation 
methods  to  more  directly  address  the  Army's  C4I  research  issues  identified  in  this  report. 

CONCLUSIONS  ON  METHOD  INTEGRATION 

This  section  describes  how  the  training  and  evaluation  methods  recommended  in  this  report, 
in  concert  with  digital  technologies,  might  help  design  and  develop  a  digital  training  environment. 
Key  training  and  evaluation  considerations  for  this  environment  focus  on  the  need  to  pictorially 
communicate  the  battlefield  situations  depicted  on  digital  displays. 

Design  and  Develop  a  Digital  Training  Environment 

By  design,  a  digital  training  environment  is  the  recommended  medium  for  integrating  and 
implementing  the  methods  recommended.  Figure  2  provides  an  overview  of  how  methods  directed 
at  the  pictorial  communication  of  battlefield  situations  on  C4I  displays  might  be  integrated  in  a 
digital  training  environment.  This  figure  portrays  the  use  of  a  unit-level  digital  communications 
model  linked  to  simulation  drivers,  virtual  (ModSAF)  and  constructive  (Janus),  and  C4I 
workstations  for  individual,  small  group,  and  collective  training. 

Although  a  digital  training  environment  should  entail  all  forms  of  Army  training  simulation, 
the  discussion  here  concentrates  on  the  use  of  virtual  simulation.  This  emphasis  on  virtual 
simulation  is  consistent  with  the  methods  recommended  for  defining,  communicating  and 
maintaining  a  common  picture  of  the  battlefield  on  C4I  displays.  Moreover,  the  soldier-in-the-loop 
nature  of  virtual  simulation  affords  the  human  interaction  for  meaningful  training. 

The  conclusions  on  method  integration  stress  that  the  digital  nature  of  virtual  simulation 
linked  to  C4I  displays  provides  an  environment  that  may  effectively  synthesize  training  and 
evaluation.  First,  this  section  describes  how  the  methods  recommended  and  a  digital  training 
environment  support  training.  Key  considerations  include  the  identification  of  training 
requirements,  training  development  and  delivery,  and  training  analysis  and  feedback.  Measurement 
is  essential  to  training  analysis,  feedback,  and  evaluation.  The  methods  recommended  have 
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Figure  2.  Integration  of  recommended  training  and  evaluation  methods  in  a  digital  training 
environment. 


stressed  an  empirical  approach  to  training  based  on  instrumented  C4I  systems.  Virtual  simulation’s 
unique  potential  for  training  and  evaluation  is  noted  by  Fletcher  (1994): 

By  instrumenting  the  electronic  battlefield  and  issuing  in  an  accessible,  digital  format, 
comprehensive  and  absolutely  accurate  data  on  the  physical  characteristics  and  actions  of 
entities  participating  in  an  emerging  task  situation  and  by  providing  powerful  new  display 
capabilities... that  can  replay  as  often  as  necessary  and  from  any  desired  viewpoint  the 
behavior  of  all  the  collectives  involved,  networked  simulation  both  enhances  the  most 
promising  of  our  measures  of  collective  behavior  and  makes  them  practicable.  It  provides 
the  foundation  for  a  measurement  system  that  should  substantially  advance  our  assessments 
of  crews,  teams  and  units  in  both  military  and  nonmilitary  settings,  (p.  268) 
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Finally,  this  section  describes  how  the  methods  recommended  and  a  digital  training  environment 
support  performance  evaluation.  Key  considerations  include  machine,  soldier,  and  soldier-machine 
performance. 


Training  Considerations 

This  section  describes  how  the  methods  recommended  in  this  report  might  support  training 
considerations  for  the  design  and  development  of  a  digital  training  environment. 

Training  Requirements  and  Training  Development 

Training  requirement  identification  for  defining,  communicating,  and  maintaining  a 
common  picture  of  the  battlefield  on  C4I  displays  is  a  key  aspect  of  the  methods  recommended  for 
training  pictorial  communication.  Implementation  of  these  methods  in  a  digital  training 
environment  should  refine  training  requirements  and  training  development,  and  foster  the 
environment.  Methods  for  defining  the  common  picture,  for  example,  addressed  determination  of 
the  METT-T  informational  requirements  of  combatants  and  supporters  as  a  basis  for  identifying 
C4I-based  training  requirements.  These  methods  primarily  elicited  expert’s  knowledge  on  the 
informational  requirements  for  duty  positions  within  a  battalion  or  brigade.  Knowledge  elicitation 
was  based  on  a  process  trace  of  soldier  performance  during  FXXITP  exercises  conducted  in  virtual 
and  constructive  simulation. 

Methods  for  communicating  the  picture  depicted  on  C4I  displays  stressed  the  training 
requirement  to  shape  the  mental  model  of  trainees  to  better  understand  the  battlefield 
representations  depicted.  These  methods  identified  relatively  unmet  requirements  to  train 
battlefield  pattern  and  situation  recognition,  and  their  C4I  representations  in  particular. 
Implementation  of  these  methods  should  result  in  a  detailed  analysis  of  the  types  of  battlefield 
patterns  and  situations  that  should  be  trained.  The  training  methods  recommended,  such  as 
consistent  practice,  include  guidelines  on  how  that  training  should  and  should  not  be  developed. 

Methods  for  maintaining  the  battlefield  picture  depicted  on  C4I  displays  also  addressed 
training  requirements  based  on  an  expert  analysis  of  a  unit’s  informational  requirements.  That 
analysis  should  provide  relatively  precise  specifications  of  training  requirements,  to  include  the 
type  of  METT-T  information  required  by  each  duty  position  and  the  information  exchanges 
required  to  maintain  a  common  picture  across  the  unit.  Method  background  also  identified  the 
training  requirement  to  provide  a  mental  model  for  managing  information  that  reflects  the 
communication  capabilities  of  their  actual  C4I  systems  and  their  unit’s  digital  architecture. 

The  methods  recommended  how  a  communications  model  could  be  developed  to  help  form  mental 
models  of  information  exchange  at  individual,  small  group,  and  collective  levels. 

Training  Delivery 

This  report’s  background  discussed  how  digital  technologies  impact  the  pillars  of  Army 
training-the  institution,  the  unit  and  self-development.  A  major  shift  in  training  delivery  pillars  is 
occurring  based  on  the  ability  of  digital  technologies  to  provide  distance  learning  and  create  a 


57 


virtual  classroom.  Enabling  assumptions  are  that  digital  technologies  can  deliver  training  to  any 
individual  at  any  location,  and  simulate  a  work  context  such  as  a  battlefield  situation.  Notably,  this 
“environment”  without  walls  includes  a  mix  of  constructive,  virtual,  and  live  simulations.  How  a 
digital  training  environment  might  balance  concerns  about  training  effectiveness  and  efficiency,  the 
benefits  and  costs  of  delivering  customized  training,  was  also  discussed. 

The  recommended  methods  for  communicating  a  common  picture  stressed  how  digital 
technologies  can  represent  and  link  the  trainee’s  mental  model  with  the  microworld  environments 
reflected  in  C4I  displays  and  virtual  simulation.  The  recommended  pattern  and  situation 
recognition  methods  employ  these  technologies  to  deliver  training  in  the  form  of  consistent  practice 
to  improve  performance.  Other  methods  leverage  the  ability  of  digital  technologies  to  enable 
learner-controlled  transformations  and  augmentations  of  microworld  representations  to  help 
trainees  understand  and  respond  to  the  battlefield  situations  depicted  on  C4I  displays. 

The  methods  for  maintaining  a  common  picture  were  deliberately  centered  in  a  digital 
training  environment  (see  Figure  2).  Methods  for  individual,  small  group,  and  collective  training 
using  the  CPCM  communications  model  were  described.  Methods  for  delivering  this  training 
stressed  formats  that  depict  C4I  display  products,  and  mimic  the  process  of  information  exchange 
by  which  those  products  are  created  and  maintained. 

Training  Analysis  and  Feedback 

The  training  methods  recommended  consistently  employ  digital  technologies  to  provide 
training  analysis  and  feedback  for  pictorial  communication  training.  Virtual  simulation,  in 
particular,  establishes  a  comprehensive  database  on  the  actions  of  simulated  entities  participating  in 
complex  and  dynamic  training  exercises  (Fletcher,  1994).  Instrumented  C4I  systems  establish  a 
comprehensive  database  on  the  information  exchanges  of  real  soldiers  that  result  in  battlefield 
depictions  on  their  tactical  displays.  Coupled  in  a  digital  training  environment,  these  technologies 
provide  an  unprecedented  capability  to  automatically  record  and  analyze  training  performance,  and 
to  develop  and  provide  objective  feedback. 

A  summary  indication  of  how  digital  methods  might  provide  training  analysis  and  feedback 
is  the  characterization  of  the  battlefield  situation  depicted  on  C4I  displays  as  a  meaningful, 
measurable,  and  collaborative  product.  A  composite  database  of  the  elements  depicted  on  the 
tactical  display  by  source  and  path  represents  a  measurable  product  achieved  through  collaborative 
information  processing  and  management.  The  methods  recommended  repeatedly  apply  this  C4I 
database  to  analyze  individual,  small  group,  and  collective  training  exercises.  The  focus  on 
pictorial  communication  of  C4I-based  battlefield  situations  exploits  this  database  to  generate 
objective  training  feedback  at  both  a  process  and  product  level.  A  working  example  of  product 
feedback  was  automated  snapshot  comparisons  between  the  C4I  displays  of  various  unit  members 
that  might  disclose  important  discrepancies  in  the  battlefield  situations  depicted. 

These  recommended  methods  were  also  designed  to  provide  training  feedback  that  is 
meaningful.  The  METT-T  organization  of  methods  may  serve  as  a  meaningful  framework  for 
structuring  feedback  in  a  manner  that  reinforces  and  shapes  the  mental  models  of  military  trainees. 
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Methods  for  communicating  the  common  picture  stressed  analysis  of  a  trainee’s  component  skills, 
such  as  pattern  recognition,  to  build  situation  recognition  and  response  skills.  Methods  for 
maintaining  the  common  picture  were  designed  to  assess  and  promote  an  understanding  of  how 
trainees’  performance  contributes  to  the  unit’s  informational  requirements,  and  overall  mission 
accomplishment.  Feedback  methods  on  information  management  performance,  for  example, 
pictorially  highlighted  training  deficiencies  at  both  the  process  and  product  level.  Moreover,  all  of 
the  methods  recommended  were  grounded  in  representative  FXXITP  training  exercises  and  actual 
or  simulated  C4I  systems  to  provide  a  meaningful  context  for  training. 

Evaluation  Considerations 


This  section  describes  how  the  methods  recommended  in  this  report  might  support 
evaluation  considerations  for  the  design  and  development  of  a  digital  training  environment. 

A  digital  training  environment  should  render  feedback  mechanisms  to  evaluators  and  bill 
payers  to  justify  the  training  and  environment  provided  (Deitchman,  1993).  The  digital 
technologies  that  comprise  this  environment  should  enable  the  controls  required  to  develop 
systematic  procedures  and  standardized  training  conditions  to  evaluate  performance.  This 
discussion  of  performance  evaluation  in  a  digital  training  environment  is  limited.  The  intent  is  to 
indicate  how  the  methods  recommended  for  training  pictorial  communication  on  C4I  displays  apply 
to  performance  evaluation.  The  recommended  methods  target  the  collaborative  potential  of  soldier- 
machine  systems.  The  emphasis  here  is  that  performance  evaluation,  made  possible  by  a  digital 
training  environment,  is  essential  to  achieving  that  potential. 

Machine  Performance 


Training  on  C4I  systems  is  directly  dependent  upon  the  functional  capabilities  and  operating 
procedures  associated  with  this  equipment.  Training  requirements  and  programs  for  C4I  systems 
are  best  based  on  a  detailed  understanding  of  the  required  interactions  between  soldiers  and  C4I 
machines.  A  more  expansive  notion  of  a  digital  training  center,  therefore,  might  include 
evaluations  of  machine  performance  to  determine  capabilities  and  support  training  (Coe  et  al., 
1997). 


The  need  for  more  precise  specification  and  testing  in  the  development  of  C4I  systems  was 
reflected  by  the  call  for  MOEs  on  hardware  performance  (J.  Hiller,  personal  communication, 
December  9, 1996).  The  functional  description  for  FBCB2also  stresses  objective  assessments  of 
machine  performance: 

...  An  objective  assessment  of  the  information  management  process  at  each  OPFAC 
[operational  facility]  using  a  standard  set  of  analysis  tools  which  eliminates  any  subjective 
variations  in  defining  the  operational  requirements  for  that  specific  OPFAC.  A  quantifiable 
assessment  of  how  this  information  management  function  is  to  be  performed  in  terms  of 
specific  operating  parameters.  (U.S.  Army  Armor  Center,  1997,  p.  31) 
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Method  recommendations  for  developing  C4I  display  codes  may  result  in  MOEs  reflecting 
machine  performance,  and  directing  machine  performance  improvements.  Such  measures  should 
provide  an  objective  basis  for  identifying  and  quantifying  met  and  unmet  C4I  requirements,  and 
help  substantiate  the  impact  of  requested  improvements.  The  METT-T  organization  of  these  C4I 
display  codes  may  provide  a  meaningful  structure  for  evaluating  and  reporting  C4I  system 
performance.  Related  methods  for  evaluating  C4I  system  performance  in  virtual  simulation  are  also 
recommended  (Heiden,  Sever,  Smith  &  Throne,  1996). 

Soldier  Performance 


Evaluations  of  soldier  performance  as  a  result  of  training  are  required  to  assess  and  improve 
the  training  provided.  More  global  evaluations  of  soldier  performance  are  needed  to  assess  training 
effectiveness  and  efficiency,  particularly  in  terms  of  cost  tradeoffs.  More  directed  evaluations  of 
soldier  performance  are  needed  to  improve  training  in  terms  of  skill  acquisition,  retention,  and 
sustainment.  While  these  soldier  performance  evaluation  concerns  are  common  to  military  training 
programs,  the  Army’s  growing  reliance  on  C4I  systems  reinforces  and  extends  these  concerns. 

The  recommended  methods  repeatedly  stress  an  empirical  approach  to  training  analysis  that 
should  directly  support  evaluation  concerns.  The  core  to  this  approach  is  the  composite  database  of 
information  elements  depicted  on  the  tactical  display  that  provides  tractable  links  to  this  product, 
and  the  soldier  performance  process  by  which  this  product  is  generated  and  maintained. 
Implementation  of  these  methods  with  instrumented  C4I  systems  in  virtual  simulation  should 
overcome  many  of  the  limitations  associated  with  observing  and  measuring  soldier  performance, 
particularly  in  combat  vehicle  settings.  While  constructive  simulation  requires  input  data  on 
soldier  performance,  virtual  simulation  affords  this  data  via  soldier-in-the-loop  performance 
(Deitchman,  1993).  Virtual  simulation  also  provides  a  powerful  set  of  tools  and  utilities  for 
structuring  evaluation  conditions  to  attain  more  robust  findings  on  training  efficiency  and 
effectiveness. 

Soldier-Machine  Performance 

Perhaps  an  ultimate  function  of  a  digital  training  environment  is  to  support  the  evaluation  of 
soldier-machine  performance.  Evaluation  of  a  complete  system,  soldiers  and  machines,  might 
appear  to  exceed  the  role  of  an  environment  dedicated  to  training.  This  notion  of  a  digital  training 
environment  is  admittedly  broad  and  pragmatic;  there  may  be  no  practical  alternative  for  such 
evaluations. 

These  methods  accentuate  the  cumulative  effects  in  soldier-machine  performance 
anticipated  by  coordinated  improvements  in  both  soldier  and  machine  performance.  Methods 
recommended  for  knowledge  elicitation,  for  example,  were  designed  to  match  machine 
characteristics  to  soldiers,  to  shape  C4I  representations  to  soldiers’  mental  models.  Training 
methods  for  communicating  the  common  picture  were  designed  to  match  soldier  characteristics  to 
machines,  to  shape  soldiers’  mental  models  to  microworld  C4!  representations.  A  summary 
example  of  this  emphasis  on  collaborative  performance  is  the  characterization  of  soldiers  and  C4I 
systems  as  joint  cognitive  systems.  The  training  methods  recommended  entail  objective  and 
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automated  measures  for  evaluating  the  products  and  process  of  soldier-machine  interaction.  These 
methods  and  measures  should  support  evaluations  of  soldier-machine  performance  in  an 
environment  that  provides  realistic  battlefield  settings.  For  a  programmatic  example  of  how  virtual 
simulation  can  be  used  to  evaluate  soldier-machine  performance  as  a  function  of  C4I  systems,  see 
Leibrecht  et  al.  (1994). 

SUMMARY 

The  U.S.  Army  is  focused  on  information-age  warfare  and  the  exploitation  of  information 
technologies  to  maintain  a  dominant  force.  At  a  visionary  level,  this  is  a  force  of  cyber  warriors 
with  humans  and  computers  allied  as  a  joint  cognitive  system.  An  objective  that  epitomizes  this 
exploitation  is  the  provision  of  a  common  and  relevant  picture  of  the  battlefield  situation  to  all 
warfighters  and  supporters.  Training  and  evaluation  methods  to  improve  soldier-computer 
performance  are  needed  to  achieve  the  Army's  information  age  objectives.  This  report  identifies  a 
set  of  research  issues  and  provides  corresponding  method  recommendations  to  help  realize  the 
potential  of  C4I  systems  to  train  and  empower  an  information-age  force. 

The  approach  was  based  on  several  key  assumptions.  First,  computers  can  and  must  help 
solve  many  of  the  training  problems  they  create.  Second,  Cl  displays  should  convey  needed 
battlefield  information  in  picture  formats  the  user  can  satisfactorily  receive,  understand,  and  act  on. 
While  textual  and  tabular  formats  may  provide  additional  detail,  a  pictorial  representation  of  the 
battlefield  situation  is  a  foremost  concern.  Third,  when  a  worker's  tool  is  an  information 
technology,  such  as  a  Cl  system,  communication  between  that  tool  and  the  worker  is  critical  to 
their  collaborative  performance.  These  assumptions  are  introduced  in  this  report's  title  and 
maintained  throughout  the  report's  method  recommendations  for  training  and  evaluation. 

A  final  assumption  is  that  measurement  is  essential  to  training  and  evaluation.  However, 
direct  observation  and  objective  measurement  of  performance  during  military  training  exercises  is 
difficult.  A  primary  method  recommendation,  therefore,  is  to  instrument  Cl  systems  to  compile  a 
log  of  all  soldier-computer  interactions  as  an  empirical  basis  for  assessing  performance  and  giving 
feedback.  Recommendations  on  evaluation  methods  are  designed  to  provide  an  automatic 
empirical  account  of  the  pictorial  elements  required  for  the  common  picture  product  and,  the 
process  by  which  those  elements  are  created  and  maintained.  Recommendations  on  training 
methods  use  this  objective  assessment  to  provide  pictorial  feedback  on  process  and  product 
performance. 

The  report's  working  example  of  empirical  assessment  and  feedback  is  automated  pictorial 
comparisons  of  the  battlefield  situations  depicted  on  the  C4I  displays  of  any  or  all  unit  members  at 
any  time  during  a  training  exercise.  Such  comparisons  are  analogous  to  the  "compare  document" 
function  for  comparing  textual  products.  Instrumented  C4I  systems  could  readily  provide  a  similar 
"compare  picture"  function.  Automated  comparisons  of  a  company  commander’s  and  platoon 
leader’s  C4I  displays,  for  example,  might  graphically  highlight  important  discrepancies,  or  what  is 
uncommon,  in  their  depicted  battlefield  situations. 

The  background  section  reviews  some  barriers  that  deter  the  application  of  training 
technology.  These  barriers  include  a  project  versus  program  approach  to  training  development,  a 
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failure  to  apply  more  unique  medium  capabilities,  and  a  failure  to  match  training  content  to  what 
trainees  already  know.  This  review  stresses  the  relatively  unique  ability  of  digital  technologies  to 
emulate  battlefield  situations  on  a  C4I  display  and  the  mental  models  of  trainees,  and  then 
perceptually  link  these  models  to  train  pictorial  communication.  Digital  training  implications,  both 
theoretical  and  practical,  are  examined  to  balance  training  efficiency  and  effectiveness. 

The  method  section  identifies  three  main  areas  of  research  directed  at  the  pictorial 
communication  of  battlefield  situations  on  C4I  displays.  These  research  areas  are  centered  on 
common  picture  training  and  evaluation  requirements:  define,  communicate,  and  maintain  a 
common  picture  of  the  battlefield.  For  each  of  these  areas,  the  method  section  identifies  a  set  of 
research  issues  related  to  the  requirement,  and  then  recommends  a  set  of  training  and  evaluation 
methods  designed  to  help  address  the  issues  raised.  The  methods  recommended  rely  on  others 
documented  training  and  evaluation  methods.  The  contribution  of  this  report,  at  best,  is  to 
document  how  the  methods  reviewed  might  be  adapted  and  integrated  to  help  the  Army  meet  some 
important  C4I  training  and  evaluation  requirements. 

The  conclusion  section  integrates  the  training  and  research  methods  recommended  in  this 
report,  in  concert  with  digital  technologies,  to  help  design  and  develop  a  digital  training 
environment  directed  at  pictorial  communication.  This  section  stresses  that  method  integration  in 
virtual  simulation,  in  particular,  might  provide  an  environment  that  effectively  synthesizes  training 
and  evaluation.  Key  training  considerations  include  the  identification  of  training  requirements, 
training  development  and  delivery,  and  training  analysis  and  feedback.  Key  evaluation 
considerations  include  machine,  soldier,  and  soldier-machine  performance. 

The  methods  recommended  in  this  report  are  small  building  blocks  in  the  larger  body  of 
training  and  evaluation  research  and  development  required  to  achieve  the  anticipated  potential  of 
C4I  systems  and  the  Army's  modernization  objectives.  These  method  recommendations  should 
help  training  and  evaluation  researchers  improve  soldier-computer  interaction  and  foster  the  skills 
needed  to  understand  and  maintain  a  pictorial  depiction  of  a  battlefield  situation  on  a  digital 
display.  Method  implementation  will  require  coordinated  efforts  between  these  researchers  and  the 
developers  of  training,  training  simulation,  and  digital  systems  to  help  the  Army  design  and 
develop  a  digital  training  environment  that  provides  the  skills  required  for  an  information  age  force. 
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APPENDIX  A 


1SG 

AAR 

ABCS 

ARI 

Aves 

AWE 

C4I 

CCIR 

CDR 

COA 

Comm 

CPCM 

D 

DA 

Dir 

DP 

EEFI 

EPSS 

FASTTRAIN 

FBC 

FBCB2 

FFIR 

FRAGO 

FSO 

FXXITP 

INA 

Info 

ISD 

IVIS 

LTIOV 

METT-T 

METT-TC 

MIDAS 

MOA 

ModSAF 

MOE 

NTC 

OPFAC 

OPORD 

OPS 


ACRONYMS  and  ABBREVIATIONS 

First  Sergeant 

After  Action  Review 
Army  Battle  Command  System 
U.S.  Army  Research  Institute 
Avenues 

Advanced  Warfighting  Experiment 

Command,  Control,  Communication,  Computer,  and  Intelligence 

Commander’s  Critical  Information  Requirements 

Commander 

Course  of  Action 

Communicate 

Common  Picture  Communications  Model 

Dimensional 
Department  of  the  Army 
Direction 
Duty  Position 

Essential  Elements  of  Friendly  Information 
Electronic  Performance  Support  Systems 

Force  XXI  Training  Methods  and  Strategies 

Future  Battlefield  Conditions 

Force  XXI  Battle  Command  Brigade  and  Below 

Friendly  Forces  Information  Requirements 

Fragmentary  Order 

Fire  Support  Officer 

Force  XXI  Training  Program 

Information  Needs  Analysis 
Information 

Instructional  Systems  Development 
Intervehicular  Information  System 

Latest  Time  Information  of  Value 
Mission,  Enemy,  Terrain,  Troops,  and  Time 
Mission,  Enemy,  Terrain,  Troops,  Time,  and  Civilians 
Man-Machine  Integrated  Design  and  Analysis  System 
Memorandum  of  Agreement 
Modular  Semi-Automated  Forces 
Measure  of  Effectiveness 

National  Training  Center 

Operational  Facility 
Operations  Order 
Operations 


A-l 


PIR 

Priority  Information  Requirements 

Recvd 

Reqd 

ROTC 

Received 

Required 

Reserve  Officer  Training  Corps 

52 

53 

SALUTE 

SIMNET 

SME 

SOP 

Sync 

Intelligence  Officer 

Operations  and  Training  Officer 

Size,  activity,  location,  unit  type,  time  and  equipment 
Simulation  Networking 

Subject  Matter  Expert 

Standing  Operating  Procedure 

Synchronization 

IF 

TMT 

TOC 

TRADOC 

Task  Force 

Team  Model  Training/Trainer 

Tactical  Operations  Center 

Training  and  Doctrine  Command 

USAARMC 

U.S.  Army  Armor  Center 

VTP 

Virtual  Training  Program 

WARNO 

Warning  Order 

A-2 


